Image classification method and device based on HR-GAT
Through HR-GAT's hierarchical attention modeling and residual feature fusion, the problems of feature hierarchy confusion and deep network degradation in image classification are solved, the classification accuracy and stability are improved, and efficient and lightweight image classification is achieved.
Patent Information
- Application Number
- CN202510909154.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-02
- Publication Date
- 2025-10-10
AI Technical Summary
Existing graph attention networks (GATs) suffer from feature hierarchy confusion, deep network degradation, and noise sensitivity in image classification tasks, resulting in a decrease in classification accuracy. In addition, the model is highly complex and relies on a large amount of labeled data.
The hierarchical residual graph attention network (HR-GAT) is adopted to construct a graph structure by fusing hierarchical attention modeling with residual features. The multi-level residual attention mechanism and channel attention mechanism are used in combination with the support vector machine (SVM) for end-to-end optimization to extract multi-level features of the image.
It improves the classification accuracy and model stability in complex scenarios, reduces model complexity, enhances noise resistance, and achieves efficient and lightweight image classification.
Smart Images
Figure CN120766027A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of image processing and pattern recognition, in particular to an image classification method and device based on hierarchical residual graph attention network (HR-GAT). BACKGROUND
[0002] In recent years, graph neural networks (GNN) have shown unique advantages in image classification tasks by modeling the relational dependencies of non-Euclidean data structures (such as graphs, point clouds). Traditional convolutional neural networks (CNN) rely on regular local grid convolution kernels, which are difficult to effectively capture long-range semantic associations in images (such as the global relationship of scattered object components). Graph structure maps pixels to nodes and spatial or semantic associations to edges, providing a more flexible framework for modeling complex scenes. In particular, graph attention networks (GAT) significantly improve the relevance of node feature interaction through dynamic attention weight distribution. However, existing GAT-based image classification methods still have the following three bottlenecks.
[0003] Single-layer homogeneous attention mechanism cannot separate local details (such as edges, textures) from global semantics (such as object component relationships), resulting in ambiguous feature representation; as the number of network layers increases, node features gradually converge due to information transmission attenuation, resulting in a decrease in classification accuracy; global attention is easily disturbed by background pixels (such as organ overlap in medical images, and mixed ground objects in remote sensing images), resulting in weakening of key feature responses.
[0004] In the prior art, some studies attempt to alleviate the above problems through multi-channel fusion or cross-domain aggregation, but such schemes often introduce redundant calculations (such as multiple branch parameter superposition), leading to a sharp increase in model complexity, and rely on a large amount of labeled data to support training. In addition, the compatibility of traditional GAT models with downstream classifiers (such as SVM) is insufficient, and feature extraction and classification decision are fragmented, limiting the potential for end-to-end optimization.
[0005] Under this background, there is an urgent need for a lightweight image classification scheme that can retain the advantages of graph structure modeling, optimize feature granularity in layers, and enhance noise resistance, to meet the high-precision and high-efficiency classification needs in image analysis, autonomous driving environment perception, and other scenarios. SUMMARY
[0006] The present application aims to solve the technical problems of existing graph attention networks (GAT) in image classification tasks, such as feature level confusion, deep network degradation, and noise sensitivity.
[0007] The first purpose of this application is to propose an image classification method based on hierarchical residual graph attention network (HR-GAT), which significantly improves the classification accuracy and model stability in complex scenarios by fusing hierarchical attention modeling with residual features.
[0008] The second objective of this application is to propose an image classification device based on HR-GAT.
[0009] To achieve the above objectives, the first embodiment of the present application proposes an image classification method and apparatus based on HR-GAT, comprising the following steps:
[0010] Input the image to be classified;
[0011] Perform standardization and size unification preprocessing on the image data, and divide the preprocessed data into training set and test set;
[0012] Design the HR-GAT network structure and perform model training based on the HR-GAT network model and training image set;
[0013] The images in the image set are input into the trained HR-GAT multi-level attention network to extract features, and the extracted features are input into the support vector machine (SVM) classifier for classification.
[0014] Optionally, in some embodiments of the present application, the HR-GAT model includes:
[0015] Step 1: Segment the input image into superpixel regions or image blocks, construct a graph structure with superpixel regions or image blocks as nodes and the spatial relationships between adjacent superpixel regions or image blocks as edges;
[0016] Step 2: The graph structure generated in the first step is input into a multi-level residual attention network to extract features. The bottom attention layer restricts node interaction within a local neighborhood (such as a 3×3 window) and focuses on edge or texture features. The high-level attention layer allows global node interaction and models the semantic association features of objects.
[0017] Step 3: The features generated in the second step are connected between levels using residual connections to obtain fusion feature weights. The formula is shown in formula (1):
[0018] H (l+1) =σ(α·Attention(H (l) )+(1-α)·H (l) ) (1)
[0019] Among them H (l)is the input feature matrix of the l-th layer, Attention is the attention operation function, a is the learnable weight, and σ is the activation function, H (l+1) is the output feature after fusion.
[0020] Step 4: The fusion feature generated in the third step is guided into a pooling layer to generate an image-level feature vector h global , as shown in equation (2):
[0021]
[0022] where ω i is the attention weight of the i-th node, h i is the feature vector of the i-th node, N is the total number of nodes, and MLP(.) is a multi-layer perceptron function.
[0023] Optionally, in some embodiments of the present application, the first step specifically includes:
[0024] Step 1) Constructing a spatial adjacency edge for image superpixel regions or image block nodes, connecting only nodes within adjacent k1xk1 windows (e.g., k1=3);
[0025] Step 2) Using a normalized adjacency matrix to eliminate scale differences, as shown in equation (3):
[0026] A norm = D -1 / 2 AD -1 / 2 (3)
[0027] where A is the adjacency matrix and D is the degree matrix.
[0028] Optionally, in some embodiments of the present application, the method further includes introducing a channel attention mechanism in the high-level attention layer:
[0029] Performing global average pooling (GAP) on the node feature H∈R N×D to generate channel weights, as shown in equation (4),
[0030] s = σ(W2·ReLU(W1·GAP(H))) (4)
[0031] where W1 and W2 are fully connected layer parameters.
[0032] Rescale the feature channels using the weight s, as shown in equation (5),
[0033]
[0034] where is the channel-wise product.
[0035] Optionally, in some embodiments of the present application, the learnable weight α of the residual connection is optimized by:
[0036] α=sigmoid(β) (6)
[0037] Initialize α = 0.5 and dynamically adjust it through back propagation during training; constrain α∈[0,1] to prevent gradient explosion, and β is a trainable parameter.
[0038] Optionally, in some embodiments of the present application, the second step further includes applying multi-granularity attention fusion to the pooling layer:
[0039] Calculate attention weights for low-level and high-level features separately and
[0040] The weighted fusion generates the final feature vector. The process is shown in formula (7).
[0041] Where λ is the learnable fusion coefficient, is the image-level feature aggregated from the bottom attention layer, is the image-level feature aggregated from high-level attention layers.
[0042] Optionally, in some embodiments of the present application, the second step further includes adopting a joint optimization strategy for the model training:
[0043] The loss function L is the cross entropy loss L CE and feature discrimination loss L FD The weighted sum of is as shown in formula (8):
[0044] L=L CE +γ·L FD (8)
[0045] Where γ is a hyperparameter (weight factor).
[0046] The feature discrimination loss is calculated by maximizing the inter-class distance and minimizing the intra-class distance, as shown in formula (9):
[0047]
[0048] Where |.| represents the norm (such as L2 norm, i.e., Euclidean distance), h is the feature vector of the sample, y is the category label of the sample, i and j represent the indexes of two different (or identical) samples in the data set, and I is the indicator function. Optionally, in some embodiments of the present application, the decision function of the SVM classifier is as shown in formula (10),
[0049]
[0050] Where sign(.) is the sign function, N is the number of support vectors, α i is the Lagrange multiplier corresponding to the i-th support vector, y i is the category label of the i-th support vector, K(.,.) is the kernel function (KernelFunction), is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the i-th support vector, h global is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the input image x to be classified, and b is the bias term.
[0051] The kernel function K uses the radial basis kernel function weighted by attention weight, as shown in formula (11):
[0052]
[0053] Where exp(.) is the natural exponential function, ||.|| 2 is the square of the L2 norm of the vector, h i is the first input feature vector, h j is the second input feature vector, ω i is the eigenvector h i The associated attention weight, ω j is the eigenvector h j The associated attention weight, 2σ 2 is the kernel bandwidth parameter in the radial basis kernel function.
[0054] Optionally, in some embodiments of the present application, the preprocessing includes:
[0055] Normalize the input image so that the pixel value range is scaled to [0,1];
[0056] Use bilinear interpolation to unify the image size to H×W (such as 224×224);
[0057] Expand the grayscale image into pseudo RGB three channels.
[0058] To achieve the above objectives, the second embodiment of the present application proposes an image classification device based on HR-GAT, including:
[0059] Graph construction module: maps image superpixel regions or image patches into graph nodes and constructs adjacent edges;
[0060] Hierarchical attention module: performs local or global attention calculation and residual fusion;
[0061] Dynamic pooling module: generates image-level features based on attention weights;
[0062] Joint training module: synchronously optimizes HR-GAT and SVM classifier parameters;
[0063] Feature extraction module: uses a multi-level attention mechanism to extract multi-level features of the image;
[0064] Classification module: The extracted features are input into the SVM classifier for classification.
[0065] Optionally, in some embodiments of the present application, the HR-GAT network includes:
[0066] Step 1: Segment the input image into superpixel regions or image blocks, construct a graph structure with superpixel regions or image blocks as nodes and the spatial relationships between adjacent superpixel regions or image blocks as edges;
[0067] Step 2: The graph structure generated in the first step is input into a multi-level residual attention network to extract features. The bottom attention layer restricts node interaction within a local neighborhood (such as a 3×3 window) and focuses on edge or texture features. The high-level attention layer allows global node interaction and models the semantic association features of objects.
[0068] Step 3: The features generated in the second step are connected between levels using residual connections to obtain fusion feature weights. The formula is shown in formula (1):
[0069] H (l+1) =σ(α·Attention(H (l) )+(1-α)·H (l) ) (1)
[0070] Among them H (l) is the input feature matrix of the lth layer, Attention is the attention operation function, α is the learnable weight, σ is the activation function, H (l+1) The output features after fusion;
[0071] Step 4: The fusion features generated in the third step guide the pooling layer to generate the image-level feature vector h global , as shown in formula (2):
[0072]
[0073] where ω i is the attention weight of the i-th node, h i is the feature vector of the i-th node, N is the total number of nodes, and MLP(.) is the multi-layer perceptron function;
[0074] Optionally, in some embodiments of the present application, the first step specifically includes:
[0075] Step 1) construct spatial adjacency edges for image superpixel regions or image block nodes, connecting only nodes within adjacent k1×k1 windows (e.g., k1=3);
[0076] Step 2) Use the normalized adjacency matrix to eliminate scale differences, as shown in formula (3):
[0077] A norm =D -1 / 2 AD -1 / 2 (3)
[0078] Where A is the adjacency matrix and D is the degree matrix.
[0079] Optionally, in some embodiments of the present application, the method further includes introducing a channel attention mechanism in a high-level attention layer:
[0080] For node features H∈R N×D Perform global average pooling (GAP) to generate channel weights, as shown in formula (4),
[0081] s=σ(W2·ReLU(W1·GAP(H))) (4)
[0082] Among them, W1 and W2 are the parameters of the fully connected layer.
[0083] Use weight s to recalibrate the feature channel, as shown in formula (5),
[0084]
[0085] in is the channel-wise product.
[0086] Optionally, in some embodiments of the present application, the learnable weight α of the residual connection is optimized by:
[0087] α=sigmoid(β) (6)
[0088] Initialize α = 0.5 and dynamically adjust it through back propagation during training; constrain α∈[0,1] to prevent gradient explosion, and β is a trainable parameter.
[0089] Optionally, in some embodiments of the present application, the second step further includes applying multi-granularity attention fusion to the pooling layer:
[0090] Calculate attention weights for low-level and high-level features separately and
[0091] Weighted fusion generates the final feature vector:
[0092] Where λ is the learnable fusion coefficient, is the image-level feature aggregated from the bottom attention layer, is the image-level feature aggregated from high-level attention layers.
[0093] Optionally, in some embodiments of the present application, the second step further includes adopting a joint optimization strategy for the model training:
[0094] The loss function L is the cross entropy loss L CE and feature discrimination loss L FD The weighted sum of is as shown in formula (8):
[0095] L=L CE +γ·L FD (8)
[0096] Where γ is a hyperparameter (weight factor).
[0097] The feature discrimination loss is calculated by maximizing the inter-class distance and minimizing the intra-class distance, as shown in formula (9):
[0098]
[0099] Where |.| represents the norm (such as L2 norm, i.e. Euclidean distance), h is the feature vector of the sample, y is the category label of the sample, i and j represent the indexes of two different (or same) samples in the data set, and I is the indicator function (IndicatorFunction).
[0100] Optionally, in some embodiments of the present application, the decision function of the SVM classifier is as shown in formula (10),
[0101]
[0102] Where sign(.) is the sign function, N is the number of support vectors, α i is the Lagrange multiplier corresponding to the i-th support vector, y i is the category label of the i-th support vector, K(.,.) is the kernel function (KernelFunction), is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the i-th support vector, h global is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the input image x to be classified, and b is the bias term.
[0103] where the kernel function K is a radial basis kernel function weighted by attention weights, specifically as shown in equation (11),
[0104]
[0105] where exp(.) is the natural exponential function, ||.||2is the L2norm of a vector, h 2 is the first input feature vector, h i is the second input feature vector, ω j is the attention weight associated with the feature vector h i is the attention weight associated with the feature vector h i is the attention weight associated with the feature vector h j is the attention weight associated with the feature vector h j is the attention weight associated with the feature vector h 2 is the kernel bandwidth parameter in the radial basis kernel function.
[0106] Optionally, in some embodiments of the present application, the preprocessing includes:
[0107] normalizing the input image to scale the pixel value range to [0, 1];
[0108] unifying the image size to HxW (e.g., 224x224) using bilinear interpolation;
[0109] expanding the grayscale image to pseudo RGB three channels.
[0110] According to the technical solution of the embodiments of the present application, the input image is segmented into superpixel regions or image blocks, the superpixel regions or image blocks are taken as nodes, the spatial relationship between adjacent superpixel regions or image blocks is taken as edges to construct a graph structure and model the spatial correlation; the local details and global semantic features are extracted layer by layer through the multi-level residual attention layers of the HR-GAT network, the cross-layer information is fused by using dynamic weights; the discriminative feature vector is generated by combining the attention-guided pooling layer, and finally the optimized SVM classifier is used to realize efficient classification. This method significantly improves the classification accuracy and robustness in complex scenes while ensuring the lightweight of the model, and the training process is end-to-end optimized, has the characteristics of high classification efficiency, strong anti-noise ability and stable gradient, and is especially suitable for high-precision demand scenarios such as image analysis and industrial quality inspection.
[0111] The additional aspects and advantages of the present application will be further illustrated in the specific implementation together with the drawings, or verified by the technical effects of the actual application. BRIEF DESCRIPTION OF DRAWINGS
[0112] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
[0113] Figure 1 A flowchart of an image classification method based on HR-GAT provided by an embodiment of the present application is shown in FIG. 1.
[0114] Figure 2 A schematic diagram of an HR-GAT feature extraction model according to an embodiment of the present application is shown in FIG. 2.
[0115] Figure 3 A structural schematic diagram of an image classification device based on HR-GAT provided by an embodiment of the present application is shown in FIG. 3. DETAILED DESCRIPTION
[0116] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.
[0117] In an example of the present application, an image classification method based on a hierarchical residual graph attention network (HR-GAT) is proposed. The method models the multi-level spatial correlation and semantic dependency between image pixels, combines graph structure representation and deep learning technology, and realizes the fine differentiation of targets in complex scenes. The core idea is to convert the image into a multi-scale graph structure, dynamically capture the synergistic effect of local features and global context information using the attention mechanism, and automatically learn the essential representation of the image by computer, thereby breaking through the limitations of traditional artificial feature design and providing an image classification solution with both explainability and high efficiency.
[0118] Before introducing the specific implementation of HR-GAT, it is necessary to clarify the challenges of traditional graph neural networks and existing deep learning models in image classification. Although traditional GCN can model non-Euclidean space relationships, it has the following key problems. A single scale of graph structure is difficult to express the multi-level semantics of the image, resulting in a split between detailed information and global context; the node feature aggregation process uses a fixed weight strategy, ignoring the differential correlation between local and global features; deep network training is prone to gradient decay, limiting the model's expression ability. The mainstream CNN is limited by the regular grid structure and is difficult to effectively model long-range dependencies, and the pooling operation may lose spatial topological information.
[0119] In view of this, this application proposes an image classification method and device based on HR-GAT (Hierarchical Residual Graph Attention Network). This method effectively improves the accuracy and efficiency of image classification by constructing a graph structure, a multi-level residual attention mechanism, a channel attention mechanism, residual connection optimization, multi-granularity attention fusion, a joint optimization strategy, and a dedicated image classification device.
[0120] The following describes an image classification method and apparatus based on HR-GAT according to an embodiment of the present invention with reference to the accompanying drawings. First, the image classification method based on HR-GAT according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0121] Figure 1 This is a flow chart of an image classification method based on HR-GAT provided in an embodiment of the present application. It should be noted that the image classification method based on HR-GAT in an embodiment of the present application can be applied to the image classification device based on HR-GAT in an embodiment of the present application. Figure 1 As shown, the HR-GAT-based image classification method may include the following steps.
[0122] In step 101, an image to be classified is input.
[0123] In step 102, the image data is pre-processed by standardization and size unification, and the pre-processed data is divided into a training set and a test set.
[0124] Optionally, after obtaining the image data, the image data may be resized and formatted to ensure that all images have the same size and format. Specifically, the processed image size is set to H*W pixels. Furthermore, the processed image data undergoes preprocessing such as denoising and smoothing. After completing these preprocessing steps, the preprocessed image data is divided into a training image set and a test image set, with the number of images in the training set being N and the number of images in the test set being M.
[0125] In step 103, the HR-GAT network structure is designed and model training is performed based on the HR-GAT network model and the training image set.
[0126] Optionally, in some embodiments of the present application, as Figure 2As shown in Figure 1, the HR-GAT model includes: the first step: dividing the input image into superpixel regions, constructing a graph structure with superpixel regions or image blocks as nodes and the spatial relationship between adjacent superpixel regions or image blocks as edges; the second step: the graph structure generated in the first step is input into the hierarchical residual attention network to extract features, the bottom attention layer restricts the nodes to interact within the local neighborhood (such as a 3×3 window) and focuses on edge or texture features; the high-level attention layer allows global node interaction to model the semantic association features of objects; the third step: the features generated in the second step are connected using inter-level residual connections to obtain the fusion feature weights, as shown in formula (1):
[0127] H (l+1) =σ(α·Attention(H (l) )+(1-α)·H (l) ) (1)
[0128] Among them H (l) is the input feature matrix of the lth layer, Attention is the attention operation function, α is the learnable weight, σ is the activation function, H (l+1) The output features after fusion;
[0129] Step 4: The fusion features generated in the third step guide the pooling layer to generate the image-level feature vector h global , as shown in formula (2):
[0130]
[0131] where ω i is the attention weight of the i-th node, h i is the feature vector of the i-th node, N is the total number of nodes, and MLP(.) is the multi-layer perceptron function;
[0132] Optionally, in some embodiments of the present application, the first step specifically includes:
[0133] Step 1) construct spatial adjacency edges for image superpixel regions or image block nodes, connecting only nodes within adjacent k1×k1 windows (e.g., k1=3);
[0134] Step 2) Use the normalized adjacency matrix to eliminate scale differences, as shown in formula (3):
[0135] A norm =D -1 / 2 AD -1 / 2 (3)
[0136] Where A is the adjacency matrix and D is the degree matrix.
[0137] Optionally, in some embodiments of the present application, the method further includes introducing a channel attention mechanism in a high-level attention layer:
[0138] For node features H∈R N×D Perform global average pooling (GAP) to generate channel weights, as shown in formula (4),
[0139] s=σ(W2·ReLU(W1·GAP(H))) (4)
[0140] Among them, W1 and W2 are the parameters of the fully connected layer.
[0141] Use weight s to recalibrate the feature channel, as shown in formula (5),
[0142]
[0143] in is the channel-wise product.
[0144] Optionally, in some embodiments of the present application, the learnable weight α of the residual connection is optimized by the following method (6):
[0145] α=sigmoid(β) (6)
[0146] Initialize α = 0.5 and dynamically adjust it through back propagation during training; constrain α∈[0,1] to prevent gradient explosion, and β is a trainable parameter.
[0147] Optionally, in some embodiments of the present application, the second step further includes applying multi-granularity attention fusion to the pooling layer:
[0148] Calculate attention weights for low-level and high-level features separately and
[0149] The weighted fusion generates the final feature vector. The process is shown in formula (7).
[0150]
[0151] Where λ is the learnable fusion coefficient, is the image-level feature aggregated from the bottom attention layer, is the image-level feature aggregated from high-level attention layers.
[0152] Optionally, in some embodiments of the present application, the second step further includes adopting a joint optimization strategy for the model training:
[0153] The loss function L is the cross entropy loss L CE and feature discrimination loss L FD The weighted sum of is as shown in formula (8):
[0154] L=L CE +γ·L FD (8)
[0155] Where γ is a hyperparameter (weight factor).
[0156] The feature discrimination loss is calculated by maximizing the inter-class distance and minimizing the intra-class distance, as shown in formula (9):
[0157]
[0158] Where |.| represents the norm (such as L2 norm, i.e., Euclidean distance), h is the feature vector of the sample, y is the category label of the sample, i and j represent the indexes of two different (or identical) samples in the data set, and I is the indicator function. Optionally, in some embodiments of the present application, the decision function of the SVM classifier is as shown in formula (10),
[0159]
[0160] Where sign(.) is the sign function, N is the number of support vectors, α i is the Lagrange multiplier corresponding to the i-th support vector, y i is the category label of the i-th support vector, K(.,.) is the kernel function (KernelFunction), is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the i-th support vector, h global is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the input image x to be classified, and b is the bias term.
[0161] The kernel function K uses the radial basis kernel function weighted by attention weight, as shown in formula (11):
[0162]
[0163] Where exp(.) is the natural exponential function, ||.|| 2 is the square of the L2 norm of the vector, h i is the first input feature vector, h j is the second input feature vector, ω i is the eigenvector h i The associated attention weight, ω j is the eigenvector h j The associated attention weight, 2σ 2 is the kernel bandwidth parameter in the radial basis kernel function.
[0164] Optionally, in some embodiments of the present application, the preprocessing includes:
[0165] normalizing the input image to scale the pixel value range to [0, 1];
[0166] unifying the image size to HxW (such as 224x224) by bilinear interpolation;
[0167] expanding the gray image to pseudo RGB three channels.
[0168] It can be understood that the images in the test image set are input into the trained HR-GAT feature extraction model to extract features, and the extracted features are input into the SVM classifier for classification. In step 104,
[0169] extracting features by a hierarchical residual attention network, inputting the images in the test image set into the trained HR-GAT feature extraction model to extract features, and inputting the extracted features into the SVM classifier for classification.
[0170] According to the HR-GAT image classification method of the embodiments of the present application, by means of the training means of the HR-GAT model and the linear SVM classifier, an image classification model based on HR-GAT can be trained. The model can perform the overall classification task, realize the full automation of image classification, is not constrained by the spatial dimension of the image, has strong universality, and has excellent, stable and reliable model performance. The convenience and accuracy of classification are greatly improved, the operation process is simplified, and the image classification is more efficient and accurate.
[0171] In order to realize the above-mentioned embodiments, the present application further provides an image classification device based on HR-GAT.
[0172] Figure 3 A structural schematic diagram of an image classification device based on HR-GAT provided by the embodiments of the present application is shown. As shown in the figure, Figure 3 The image classification device based on HR-GAT 300 includes a graph construction module 301, a hierarchical attention module 302, a dynamic pooling module 303, a joint training module 304, a feature extraction module 305, and a classification module 306.
[0173] Specifically, the graph construction module 301 is configured to obtain image data and perform superpixel segmentation on the image, taking superpixels or image blocks as nodes and the spatial relationship between adjacent superpixels or image blocks as edges to construct a graph structure.
[0174] The hierarchical attention module 302 is configured to perform local / global attention calculation and residual fusion.
[0175] The dynamic pooling module 303 is used to generate image-level features based on the attention weights.
[0176] The joint training module 304 is used to simultaneously optimize the HR-GAT network and SVM classifier parameters and model training.
[0177] The feature extraction module 305 is used to extract multi-level features of the image using a multi-level attention mechanism.
[0178] The classification module 306 is used to input the extracted features into the SVM classifier for classification.
[0179] It should be noted that the aforementioned explanation of the embodiment of the image classification method based on HR-GAT is also applicable to the image classification device based on HR-GAT in this embodiment, and will not be repeated here.
[0180] According to the HR-GAT image classification device of the embodiment of the present application, an image classification model based on HR-GAT can be obtained through the HR-GAT model and the linear SVM classifier training method, thereby realizing the overall classification task, and can be used for fully automatic classification, without being restricted by the spatial dimension of the image to be classified, with strong applicability, good model performance, and robustness, making classification not only more convenient and effective, but also simple to operate and with better performance, thereby effectively improving the accuracy and convenience of image classification.
[0181] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.
[0182] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.
[0183] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.
[0184] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.
[0185] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0186] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.
[0187] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0188] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.
Claims
1. An image classification method based on HR-GAT, characterized in that: include: Input the image to be classified; Perform standardization and size unification preprocessing on the image data, and divide the preprocessed data into training set and test set; Design the HR-GAT network structure and perform model training based on the HR-GAT network model and training image set; The images in the image set are input into the trained HR-GAT multi-level attention network to extract features, and the extracted features are input into the support vector machine (SVM) classifier for classification.
2. The image classification method based on HR-GAT according to claim 1, characterized in that The HR-GAT network includes: Step 1: Segment the input image into superpixel regions or image blocks, construct a graph structure with superpixel regions or image blocks as nodes and the spatial relationships between adjacent superpixel regions or image blocks as edges; Step 2: The graph structure generated in the first step is input into a hierarchical residual attention network to extract features. The bottom attention layer restricts node interaction within a local neighborhood (such as a 3×3 window) and focuses on edge or texture features. The high-level attention layer allows global node interaction to model the semantic association features of objects. Step 3: The features generated in the second step are connected between levels using residual connections to obtain fusion feature weights. The formula is shown in formula (1): H (l+1) =σ(α·Attention(H (l) )+(1-α)·H (l) ) (1) Among them H (l) is the input feature matrix of the lth layer, Attention is the attention operation function, α is the learnable weight, σ is the activation function, H (l+1) is the output feature after fusion; Step 4: The fusion features generated in the third step guide the pooling layer to generate the image-level feature vector h global , as shown in formula (2): where ω i is the attention weight of the i-th node, h i is the feature vector of the i-th node, N is the total number of nodes, and MLP(.) is the multi-layer perceptron function.
3. The HR-GAT network according to claim 2, characterized in that: The neighborhood interaction range of the bottom attention layer is defined by the following steps: Step 1) construct spatial adjacency edges for image superpixel regions or image block nodes, connecting only nodes within adjacent k1×k1 windows (e.g., k1=3); Step 2) Use the normalized adjacency matrix to eliminate scale differences, as shown in formula (3): A norm =D -1 / 2 AD -1 / 2 (3) Where A is the adjacency matrix and D is the degree matrix.
4. The HR-GAT network according to claim 2, characterized in that: The high-level attention layer introduces the channel attention mechanism: For node features H∈R N×D Perform global average pooling (GAP) to generate channel weights, as shown in formula (4), s=σ(W2·ReLU(W1·GAP(H))) (4) Among them, W1 and W2 are the parameters of the fully connected layer. Use weight s to recalibrate the feature channel, as shown in formula (5), in is the channel-wise product.
5. The HR-GAT network according to claim 2, characterized in that: The learnable weight α of the residual connection is optimized by formula (6): a=sigmoid(β) (6) Initialize α = 0.5 and dynamically adjust it through back propagation during training; constrain α∈[0,1] to prevent gradient explosion, and β is a trainable parameter.
6. The HR-GAT network according to claim 2, characterized in that: The pooling layer adopts multi-granularity attention fusion, Calculate attention weights for low-level and high-level features separately and The weighted fusion generates the final feature vector. The process is shown in formula (7). Where λ is the learnable fusion coefficient, is the image-level feature aggregated from the bottom attention layer, is the image-level feature aggregated from high-level attention layers.
7. The image classification method based on HR-GAT according to claim 1, characterized in that: The model training adopts a joint optimization strategy: The loss function L is the cross entropy loss L CE and feature discrimination loss L FD The weighted sum of is as shown in formula (8): L=L CE +γ·L FD (8) Where γ is a hyperparameter (weight factor). The feature discrimination loss is calculated by maximizing the inter-class distance and minimizing the intra-class distance, as shown in formula (9): Where |.| represents the norm (such as L2 norm, i.e. Euclidean distance), h is the feature vector of the sample, y is the category label of the sample, i and j represent the indexes of two different (or same) samples in the data set, and I is the indicator function (IndicatorFunction).
8. The image classification method based on HR-GAT according to claim 1, characterized in that: The decision function of the SVM classifier is shown in formula (10), Where sign(.) is the sign function, N is the number of support vectors, α i is the Lagrange multiplier corresponding to the i-th support vector, y i is the category label of the i-th support vector, K(.,.) is the kernel function (KernelFunction), is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the i-th support vector, h global is the global image-level feature vector (extracted by the HR-GAT network) corresponding to the input image x to be classified, and b is the bias term. The kernel function K uses the radial basis kernel function weighted by attention weight, as shown in formula (11): Where exp(.) is the natural exponential function, ||.|| 2 is the square of the L2 norm of the vector, h i is the first input feature vector, h j is the second input feature vector, ω i is the eigenvector h i The associated attention weight, ω j is the eigenvector h j The associated attention weight, 2σ 2 is the kernel bandwidth parameter in the radial basis kernel function.
9. The image classification method based on HR-GAT according to claim 1, characterized in that: The pretreatment includes: Normalize the input image so that the pixel value range is scaled to [0,1]; Use bilinear interpolation to unify the image size to H×W (such as 224×224); Expand the grayscale image into pseudo RGB three channels.
10. An image classification device based on HR-GAT, characterized in that: include: Graph construction module: maps image superpixel regions or image patches into graph nodes and constructs adjacent edges; Hierarchical attention module: performs local or global attention calculation and residual fusion; Dynamic pooling module: generates image-level features based on attention weights; Joint training module: synchronously optimizes the parameters of the HR-GAT network and the SVM classifier; Feature extraction module: uses a multi-level attention mechanism to extract multi-level features of the image; Classification module: The extracted features are input into the SVM classifier for classification.