A semantic image segmentation method based on graph convolutional networks

Abstract

This invention discloses an image semantic segmentation method based on graph convolutional networks, comprising the following steps: acquiring raw image data and preprocessing it; extracting features to generate multi-scale feature maps; dividing regions based on the multi-scale feature maps, defining region units as graph nodes and constructing node feature sets; constructing an initial adjacency matrix based on the spatial proximity and feature similarity of region units; adaptively updating the initial adjacency matrix, adjusting edge weights based on semantic similarity and spatial connectivity between nodes; inputting the adaptive adjacency matrix and node feature sets into a graph convolutional network for graph convolution operations and feature aggregation; fusing the node feature representations with the multi-scale feature maps to obtain fused features; upsampling and pixel-level mapping of the fused features to generate a semantic segmentation result map. This invention integrates graph convolutional networks and multi-layer attention mechanisms to achieve adaptive modeling for image semantic segmentation, possessing advantages such as high accuracy, strong robustness, and global perception.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image semantic segmentation method based on graph convolutional networks. Background Technology

[0002] Existing image semantic segmentation methods primarily rely on convolutional neural networks or their improved structures. These networks extract spatial and semantic features from images through multi-layer convolution and downsampling operations, and then restore spatial resolution using feature fusion and upsampling. However, because convolutional operations inherently depend on local receptive fields, they struggle to effectively model the global relationships between distant regions in an image, leading to decreased segmentation accuracy in complex scenes or when similar textures are present. Furthermore, traditional feature fusion processes typically employ static weights or fixed-level concatenation, lacking adaptive optimization capabilities for features at different semantic levels.

[0003] To address this, some studies have introduced graph convolutional networks to model pixel- or region-level relational structures. However, existing solutions often employ fixed adjacency matrices or single feature relationships, failing to adequately characterize the dynamic semantic and spatial relationships between nodes in the graph. Furthermore, the lack of effective cross-feature channel attention mechanisms and global feature enhancement methods during graph convolutional aggregation leads to insufficient node feature updates and inadequate global context understanding, thus limiting the refined expression and robustness of semantic segmentation results.

[0004] Therefore, how to provide an image semantic segmentation method based on graph convolutional networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an image semantic segmentation method based on graph convolutional networks. This invention integrates graph convolutional networks and multi-layer attention mechanisms to achieve adaptive modeling of image semantic segmentation, which has the advantages of high accuracy, strong robustness and global perception.

[0006] An image semantic segmentation method based on graph convolutional networks according to an embodiment of the present invention includes the following steps:

[0007] Obtain the raw image data, perform preprocessing, and obtain a standardized image tensor;

[0008] Feature extraction is performed on the standardized image tensor to capture feature information at different spatial and semantic levels layer by layer, generating multi-scale feature maps.

[0009] Region partitioning is performed based on multi-scale feature maps. The partitioned region units are represented as graph nodes, and a node feature vector is assigned to each graph node to construct a node feature set.

[0010] Based on the spatial proximity and feature similarity relationships between regional units, the connection strength between regional units is calculated, and an initial adjacency matrix is ​​constructed.

[0011] The initial adjacency matrix is ​​adaptively updated. The edge weight adjustment value is calculated based on the semantic similarity and spatial connectivity between node feature vectors. The adjacency relationship is corrected according to the adjustment result to generate an adaptive adjacency matrix.

[0012] The adaptive adjacency matrix and the set of node features are input into the graph convolutional network, where graph convolution operations and feature aggregation are performed to output node feature representations enhanced by global context.

[0013] The node feature representations and multi-scale feature maps are subjected to feature-level concatenation and weighted fusion to obtain fused features;

[0014] The fused features are upsampled and mapped at the pixel level, and the region-level features are reconstructed to the original image resolution according to the spatial coordinates, generating a semantic segmentation result map that corresponds to the original image in terms of spatial size.

[0015] Optionally, the preprocessing includes image normalization, size normalization, channel alignment, noise suppression, data augmentation, and tensor quantization.

[0016] Optionally, the generation of the multi-scale feature map includes: performing multi-layer convolution processing on the standardized image tensor; using a sliding convolution kernel to perform weighted summation calculation on local regions within the spatial domain; extracting the local feature distribution under different receptive fields; sequentially performing nonlinear activation and batch normalization operations after the output of each convolutional layer; performing amplitude constraint and scale normalization processing on the feature response values; performing downsampling operations between adjacent convolutional layers; compressing the spatial dimension and retaining the main response region features through pooling operations; stacking the feature maps output by different levels of convolution in the channel dimension; performing feature fusion operations according to the hierarchical order; performing weighted superposition processing on high-level features and low-level features; and weighting and combining the channel responses of each layer of feature maps during the fusion process, with the weight values ​​dynamically allocated according to the relative proportion of the feature response intensity of each layer, thereby generating a multi-scale feature map.

[0017] Optionally, the construction of the node feature set includes: performing spatial partitioning operations on the multi-scale feature map to divide the image into multiple regional units, each regional unit consisting of a spatially continuous set of pixels; extracting the corresponding feature response values ​​for each regional unit; calculating the feature mean and variance of all pixels within the regional unit on each channel; obtaining the statistical feature vector of the corresponding regional unit through a weighted average; defining each regional unit as a graph node; assigning a unique node index to each graph node and establishing an index mapping relationship; extracting the center coordinate position parameters within the region corresponding to each graph node; performing channel concatenation and normalization processing with the statistical feature vector to generate a node feature vector; and arranging and aggregating all node feature vectors according to the node index order to obtain the node feature set.

[0018] Optionally, the construction of the initial adjacency matrix specifically includes:

[0019] Based on the center coordinate position parameters and statistical feature vectors of each graph node in the node feature set, the spatial distance and feature similarity between any two graph nodes are calculated. The spatial distance is obtained by taking the square root of the sum of the squares of the differences in the node center coordinates, and the feature similarity is calculated by the ratio of the inner product and the product of the modulus between the node feature vectors.

[0020] The spatial distance and feature similarity are normalized, the spatial distance is mapped to distance weight, and the feature similarity is mapped to similarity weight. The two types of weights are then weighted and combined according to a preset balance coefficient to generate a connection strength matrix.

[0021] Valid connections between nodes are filtered based on the numerical threshold of the connection strength matrix. Connections below the threshold are assigned zero weights, while connections above the threshold retain their weights, thus obtaining preliminary adjacency relationships.

[0022] Establish a matrix row and column mapping relationship based on the graph node index, fill the connection strength value of each node pair in the initial adjacency relationship into the corresponding matrix element position, fill the unconnected node pairs with zero values, and construct the initial adjacency matrix.

[0023] Optionally, the generation of the adaptive adjacency matrix specifically includes:

[0024] Based on the initial adjacency matrix and node feature set, the similarity of the node feature vectors of each graph node is calculated to obtain the semantic similarity matrix. The similarity is obtained by calculating the cosine value between any two node feature vectors, where the cosine value is the ratio of the inner product of the two node feature vectors to the product of their respective magnitudes.

[0025] Calculate the spatial distance between nodes based on the center coordinate position parameters of each node in the graph, and then normalize the distance by taking its reciprocal to obtain the spatial connectivity matrix.

[0026] The semantic similarity matrix and the spatial connectivity matrix are weighted and fused according to the weighted fusion coefficients. The fusion result is then linearly normalized to generate an edge weight adjustment matrix. The weighted fusion coefficients are adaptively adjusted according to the node feature response intensity.

[0027] The edge weights at corresponding positions in the initial adjacency matrix are corrected based on the element values ​​of the edge weight adjustment matrix. The difference matrix of the corrected edge weights is calculated, and the difference matrix is ​​weighted and superimposed to generate the updated adjacency relation matrix.

[0028] The updated adjacency matrix is ​​normalized to maintain a stable numerical distribution, and a threshold pruning operation is performed on all edge weights to generate an adaptive adjacency matrix.

[0029] Optionally, the output of the globally context-enhanced node feature representation specifically includes:

[0030] An adaptive adjacency matrix and a set of node features are input into a graph convolutional network. Using the adaptive adjacency matrix as an update reference, the spatial adjacency matrix and the semantic adjacency matrix are parametrically learned and adjusted to generate a spatial association matrix and a semantic relation matrix. The graph convolutional network adopts a DGCN structure, and the spatial adjacency matrix and the semantic adjacency matrix in the original DGCN network are parametrically learned. A cross-attention module is added between the spatial graph convolutional unit and the semantic graph convolutional unit, and a global attention fusion module is added at the output of the dual branch.

[0031] In the spatial graph convolutional unit, the feature set of nodes is aggregated based on the spatial correlation matrix. The features of the corresponding neighboring nodes are aggregated according to the spatial correlation weight for each graph node to generate spatial aggregated features.

[0032] In the semantic graph convolutional unit, semantic relevance convolution processing is performed on the node feature set based on the semantic relation matrix. For each graph node, the features of the corresponding neighboring nodes are combined according to the semantic relevance weights to generate semantic aggregate features.

[0033] Spatial aggregation features and semantic aggregation features are input into the cross-attention module, and the responses of the two types of feature channels are weighted and fused to generate a fused feature representation. The cross-attention module includes a feature alignment layer, a bidirectional attention weight calculation layer and a feature coupling layer.

[0034] The fused feature representation is input into the global attention fusion module, which performs global weighted updates and recalculations on the features of the fused feature representation at the node dimension, and outputs a node feature representation enhanced by global context. The global attention fusion module includes a global feature statistics layer, a global weight allocation layer, and a feature write-back layer.

[0035] Optionally, obtaining the fused features includes: aligning the node feature representation with the multi-scale feature map in the channel dimension; performing size normalization on the features at each scale of the multi-scale feature map; performing channel splicing on the aligned multi-scale features and the node feature representation; performing weighted operations on each channel feature in the spliced ​​feature map according to a preset fusion weight; calculating the weighted channel response value at each position; performing standardization and linear transformation on the weighted channel response; constraining the feature amplitude range; and rearranging and stacking the processed feature maps according to the node index order to obtain the fused features.

[0036] Optionally, the generation of the semantic segmentation result map includes: performing an upsampling operation on the fused features, expanding the feature map spatial size to the same resolution as the original image by a preset multiple, performing interpolation calculation and convolution smoothing during the upsampling process to supplement spatial details, performing pixel-level mapping on the upsampled feature mapping, converting the channel response values ​​into semantic category probability distributions, taking the category index value corresponding to the maximum probability of the semantic category probability distribution, generating a semantic label mapping matrix, adjusting the spatial arrangement of the semantic label mapping matrix to be consistent with the coordinate order of the original image, and generating a semantic segmentation result map.

[0037] The beneficial effects of this invention are:

[0038] This invention optimizes the entire image semantic segmentation process by introducing adaptive graph structure modeling and multi-level attention fusion mechanisms based on graph convolutional networks. This optimization encompasses region-level feature abstraction, graph structure construction, and node-level semantic aggregation, significantly improving segmentation accuracy and global semantic consistency. By dividing the image into regions based on multi-scale feature extraction and representing each region as a graph node, a node feature set is established. This invention effectively expresses the spatial topological relationships and semantic distribution features within the image at the graph structure level. Its adaptive adjacency matrix update mechanism combines semantic similarity and spatial connectivity between node features to dynamically adjust the connection strength between nodes. This overcomes the limitation of traditional fixed adjacency matrices in representing feature changes, enabling the graph structure to adaptively adjust with the semantic changes of the input features, thus more accurately reflecting the semantic dependencies and spatial structure between nodes.

[0039] In the graph convolution computation stage, this invention employs an improved DGCN structure, which parameterizes and updates spatial and semantic adjacency relationships to generate spatial association matrices and semantic relationship matrices, respectively. A cross-attention module is introduced between the two graph convolution paths to perform bidirectional weighted mapping and feature coupling on spatial and semantic aggregated features, fully realizing the association learning between different feature dimensions. This design enhances the complementarity and alignment between semantic levels while ensuring the integrity of spatial structural information, thereby achieving higher feature representation capabilities and semantic discrimination accuracy. Furthermore, by using a global attention fusion module to perform global weighted updates and recalculation of the fused features, adaptive enhancement of global contextual information at the node level is achieved, improving the problem that traditional local convolutional structures struggle to capture global dependencies.

[0040] Ultimately, this invention achieves high-precision alignment between region features and the original spatial structure by fusing node features enhanced by graph convolution with multi-scale feature maps and performing upsampling and mapping operations at the pixel level. This design effectively improves segmentation performance in complex backgrounds, occluded regions, and boundary details, enhancing the model's ability to recognize different semantic categories and its robustness. Overall, this invention achieves innovative breakthroughs in feature modeling, graph structure optimization, and context fusion, significantly improving the precision and accuracy of semantic segmentation while maintaining computational efficiency, and has good application and promotion value. Attached Figure Description

[0041] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0042] Figure 1 This is a flowchart of an image semantic segmentation method based on graph convolutional networks proposed in this invention;

[0043] Figure 2 This is a schematic diagram of the graph convolutional network structure for an image semantic segmentation method based on graph convolutional networks proposed in this invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0045] refer to Figure 1-2 An image semantic segmentation method based on graph convolutional networks includes the following steps:

[0046] Obtain the raw image data, perform preprocessing, and obtain a standardized image tensor;

[0047] Feature extraction is performed on the standardized image tensor to capture feature information at different spatial and semantic levels layer by layer, generating multi-scale feature maps.

[0048] Region partitioning is performed based on multi-scale feature maps. The partitioned region units are represented as graph nodes, and a node feature vector is assigned to each graph node to construct a node feature set.

[0049] Based on the spatial proximity and feature similarity relationships between regional units, the connection strength between regional units is calculated, and an initial adjacency matrix is ​​constructed.

[0050] The initial adjacency matrix is ​​adaptively updated. The edge weight adjustment value is calculated based on the semantic similarity and spatial connectivity between node feature vectors. The adjacency relationship is corrected according to the adjustment result to generate an adaptive adjacency matrix.

[0051] The adaptive adjacency matrix and the set of node features are input into the graph convolutional network, where graph convolution operations and feature aggregation are performed to output node feature representations enhanced by global context.

[0052] The node feature representations and multi-scale feature maps are subjected to feature-level concatenation and weighted fusion to obtain fused features;

[0053] The fused features are upsampled and mapped at the pixel level, and the region-level features are reconstructed to the original image resolution according to the spatial coordinates, generating a semantic segmentation result map that corresponds to the original image in terms of spatial size.

[0054] In this embodiment, the preprocessing includes image normalization, size standardization, channel arrangement, noise suppression, data augmentation, and tensor quantization.

[0055] In this embodiment, the generation of the multi-scale feature map includes: performing multi-layer convolution processing on the standardized image tensor; using a sliding convolution kernel to perform weighted summation calculation on local regions within the spatial domain; extracting the local feature distribution under different receptive fields; sequentially performing nonlinear activation and batch normalization operations after the output of each convolutional layer; performing amplitude constraint and scale normalization processing on the feature response values; performing downsampling operations between adjacent convolutional layers; compressing the spatial dimension and retaining the main response region features through pooling operations; stacking the feature maps output by different levels of convolution in the channel dimension; performing feature fusion operations according to the hierarchical order; performing weighted superposition processing on high-level features and low-level features; and weighting the channel responses of each layer of feature maps during the fusion process, with the weight values ​​dynamically allocated according to the relative proportion of the feature response intensity of each layer, thereby generating a multi-scale feature map that includes multi-scale spatial structure information and multi-layer semantic features.

[0056] In this embodiment, the construction of the node feature set includes: performing spatial partitioning operations on the multi-scale feature map to divide the image into multiple regional units, each regional unit consisting of a spatially continuous set of pixels; extracting the corresponding feature response values ​​for each regional unit; calculating the feature mean and variance of all pixels within the regional unit on each channel; obtaining the statistical feature vector of the corresponding regional unit through weighted averaging; defining each regional unit as a graph node; assigning a unique node index to each graph node and establishing an index mapping relationship; extracting the center coordinate position parameters within the region corresponding to each graph node; performing channel concatenation and normalization processing with the statistical feature vector to generate a node feature vector; and arranging and aggregating all node feature vectors according to the node index order to obtain the node feature set.

[0057] In this embodiment, the construction of the initial adjacency matrix specifically includes:

[0058] Based on the center coordinate position parameters and statistical feature vectors of each graph node in the node feature set, the spatial distance and feature similarity between any two graph nodes are calculated. The spatial distance is obtained by taking the square root of the sum of the squares of the differences in the node center coordinates, and the feature similarity is calculated by the ratio of the inner product and the product of the modulus between the node feature vectors.

[0059] The spatial distance and feature similarity are normalized, the spatial distance is mapped to distance weight, and the feature similarity is mapped to similarity weight. The two types of weights are then weighted and combined according to a preset balance coefficient to generate a connection strength matrix.

[0060] Valid connections between nodes are filtered based on the numerical threshold of the connection strength matrix. Connections below the threshold are assigned zero weights, while connections above the threshold retain their weights, thus obtaining preliminary adjacency relationships.

[0061] Establish a matrix row and column mapping relationship based on the graph node index, fill the connection strength value of each node pair in the initial adjacency relationship into the corresponding matrix element position, fill the unconnected node pairs with zero values, and construct the initial adjacency matrix.

[0062] In this embodiment, the generation of the adaptive adjacency matrix specifically includes:

[0063] Based on the initial adjacency matrix and node feature set, the similarity of the node feature vectors of each graph node is calculated to obtain the semantic similarity matrix. The similarity is obtained by calculating the cosine value between any two node feature vectors, where the cosine value is the ratio of the inner product of the two node feature vectors to the product of their respective magnitudes.

[0064] Calculate the spatial distance between nodes based on the center coordinate position parameters of each node in the graph, and then normalize the distance by taking its reciprocal to obtain the spatial connectivity matrix.

[0065] The semantic similarity matrix and the spatial connectivity matrix are weighted and fused according to the weighted fusion coefficients. The fusion result is then linearly normalized to generate an edge weight adjustment matrix. The weighted fusion coefficients are adaptively adjusted according to the node feature response intensity.

[0066] The edge weights at corresponding positions in the initial adjacency matrix are corrected based on the element values ​​of the edge weight adjustment matrix. The difference matrix of the corrected edge weights is calculated, and the difference matrix is ​​weighted and superimposed to generate the updated adjacency relation matrix.

[0067] The calculation of the difference matrix includes: performing element-by-element subtraction on the corresponding elements of the initial adjacency matrix and the edge weight adjustment matrix, arranging the resulting edge weight differences in the order of node indices to obtain the difference matrix, and normalizing and setting invalid edges to zero on the difference matrix.

[0068] The updated adjacency matrix is ​​normalized to maintain a stable numerical distribution, and a threshold pruning operation is performed on all edge weights to generate an adaptive adjacency matrix.

[0069] In this embodiment, the output of the node feature representation enhanced by global context specifically includes:

[0070] An adaptive adjacency matrix and a set of node features are input into a graph convolutional network. Using the adaptive adjacency matrix as an update reference, the spatial adjacency matrix and the semantic adjacency matrix are parametrically learned and adjusted to generate a spatial association matrix and a semantic relation matrix. The graph convolutional network adopts a DGCN structure, and the spatial adjacency matrix and the semantic adjacency matrix in the original DGCN network are parametrically learned. A cross-attention module is added between the spatial graph convolutional unit and the semantic graph convolutional unit, and a global attention fusion module is added at the output of the dual branch.

[0071] The spatial adjacency matrix is ​​used to describe the spatial topological connection relationship between nodes in the graph, and the semantic adjacency matrix is ​​used to describe the semantic similarity relationship between node features.

[0072] The generation of the spatial association matrix includes: performing linear weighting operations on the matrix elements of the spatial adjacency matrix and the adaptive adjacency matrix according to the corresponding element relationship; adjusting the weights of each connection according to a preset spatial balance coefficient to generate a fusion matrix; performing normalization processing on the element values ​​of the fusion matrix to maintain a stable distribution of matrix weights; performing threshold pruning on the normalized matrix to retain valid connections above a set threshold and set invalid connections below the threshold to zero; and finally generating the spatial association matrix.

[0073] The generation of the semantic relation matrix includes: calculating the semantic correlation coefficient between the corresponding node pairs in the semantic adjacency matrix and the adaptive adjacency matrix based on the element values ​​of the corresponding node pairs; updating the element values ​​of the semantic adjacency matrix according to the correlation coefficient to obtain a semantic weighted matrix; standardizing the elements of the semantic weighted matrix to limit the range of matrix values; smoothing the standardization result to reduce abnormal weight fluctuations; performing a threshold filtering operation on the smoothed matrix to retain the connection relationship between high similarity nodes and set the connection weight between low similarity nodes to zero, thereby generating the semantic relation matrix.

[0074] In the spatial graph convolutional unit, the feature set of nodes is aggregated based on the spatial correlation matrix. The features of the corresponding neighboring nodes are aggregated according to the spatial correlation weight for each graph node to generate spatial aggregated features.

[0075] The generation of the spatial aggregation feature includes: performing a neighborhood feature weighting operation on the feature vectors of each node in the node feature set according to the association weights between nodes in the spatial association matrix; extracting feature vectors from the neighborhood node set for each node and performing weighted summation according to the corresponding spatial association weights to obtain the weighted neighborhood features of the corresponding node; arranging and stacking the weighted neighborhood features of all nodes in the order of node index to obtain the preliminary spatial feature mapping; and performing batch normalization and nonlinear activation processing on the preliminary spatial feature mapping to generate the spatial aggregation feature.

[0076] In the semantic graph convolutional unit, semantic relevance convolution processing is performed on the node feature set based on the semantic relation matrix. For each graph node, the features of the corresponding neighboring nodes are combined according to the semantic relevance weights to generate semantic aggregate features.

[0077] The generation of the semantic aggregation features includes: performing a semantically weighted convolution operation on the feature vectors of each node in the node feature set according to the semantic correlation weights between each node in the semantic relation matrix; extracting feature vectors from the semantic neighborhood node set for each node; performing weighted summation according to the corresponding semantic correlation weights to obtain semantically weighted features; concatenating the semantically weighted features of all nodes in the order of node index to obtain a semantic feature map; performing linear transformation, nonlinear activation and normalization operations on the semantic feature map in sequence; adjusting the feature channel distribution and amplitude range to generate semantic aggregation features.

[0078] Spatial aggregation features and semantic aggregation features are input into the cross-attention module, and the responses of the two types of feature channels are weighted and fused to generate a fused feature representation. The cross-attention module includes a feature alignment layer, a bidirectional attention weight calculation layer and a feature coupling layer.

[0079] The generation of the fused feature representation includes: inputting spatial aggregation features and semantic aggregation features into a cross-attention module; performing a linear transformation on the channel dimensions of the spatial aggregation features and semantic aggregation features in a feature alignment layer to map the two types of features to the same dimensional space; calculating the weighting coefficient matrix of spatial aggregation features on semantic aggregation features and the weighting coefficient matrix of semantic aggregation features on spatial aggregation features in a bidirectional attention weight calculation layer; performing a bidirectional weighted mapping operation on the feature mapping based on the weighting coefficient matrix; and performing concatenation, weighted summation and normalization processing on the bidirectional weighted features according to the channel dimensions in a feature coupling layer to generate the fused feature representation.

[0080] The bidirectional weighted mapping operation includes: using spatial aggregation features as query vectors, semantic aggregation features as key vectors and numerical vectors, calculating the attention response from space to semantic direction to obtain spatially oriented weighted features; using semantic aggregation features as query vectors, spatial aggregation features as key vectors and numerical vectors, calculating the attention response from semantic to spatial direction to obtain semantically oriented weighted features; matching spatially oriented features and semantically oriented features in the feature dimension; and performing linear combination and normalization processing on the matched features to generate bidirectional weighted features.

[0081] The fused feature representation is input into the global attention fusion module, and the features of the fused feature representation in the node dimension are globally weighted, updated and recalculated to output the node feature representation enhanced by the global context. The global attention fusion module includes a global feature statistics layer, a global weight allocation layer and a feature write-back layer.

[0082] The global weighted update and recalculation includes: inputting the fused feature representation into the global attention fusion module; performing global average convergence and global maximum convergence operations on the fused feature representation at the node dimension in the global feature statistics layer to generate global statistical features; performing linear transformation and normalization processing on the global statistical features in the global weight allocation layer to calculate the global weighting coefficients corresponding to each channel; performing a weighted mapping between the fused feature representation and the global weighting coefficients in the feature write-back layer; performing channel-by-channel weighted update and recalculation on the node features; and outputting the node feature representation.

[0083] In this embodiment, obtaining the fused features includes: aligning the node feature representation with the multi-scale feature map in the channel dimension; performing size normalization on the features at each scale of the multi-scale feature map; performing channel splicing on the aligned multi-scale features and the node feature representation; performing weighted calculation on each channel feature in the spliced ​​feature map according to a preset fusion weight; calculating the weighted channel response value at each position; performing standardization and linear transformation on the weighted channel response; constraining the feature amplitude range; and rearranging and stacking the processed feature maps according to the node index order to obtain the fused features.

[0084] In this embodiment, the generation of the semantic segmentation result map includes: performing an upsampling operation on the fused features, expanding the feature map spatial size to the same resolution as the original image by a preset multiple, performing interpolation calculation and convolution smoothing during the upsampling process to supplement spatial details, performing pixel-level mapping on the upsampled feature mapping, converting the channel response values ​​into semantic category probability distributions, taking the category index value corresponding to the maximum probability of the semantic category probability distribution, generating a semantic label mapping matrix, adjusting the spatial arrangement of the semantic label mapping matrix to be consistent with the coordinate order of the original image, and generating the semantic segmentation result map.

[0085] Example 1:

[0086] To verify the feasibility of this invention in practice, it was applied to an intelligent security monitoring system in a large urban underground complex. The system deploys high-definition cameras at underground parking areas and entrances / exits to collect real-time images of vehicles and pedestrians. Due to the complex lighting and significant reflection interference in the underground environment, traditional image segmentation methods based on convolutional neural networks are prone to missegmentation in shadow areas, areas with light interference, and overlapping occlusions. This makes it difficult to accurately distinguish vehicle outlines, ground markings, and pedestrian areas, leading to decreased accuracy in subsequent target detection and security control, severely impacting the reliability and response efficiency of the automatic monitoring system. This invention fundamentally improves upon this problem by constructing an image semantic segmentation model based on graph convolutional networks, enabling structured representation and semantic hierarchical modeling of image data in complex environments.

[0087] In practical applications, the system first inputs the images captured by the camera into the preprocessing module, where the images are normalized, channel rearranged, and noise suppressed to generate a standardized image tensor. Subsequently, the model automatically performs multi-layer convolutional feature extraction, capturing spatial structural information and semantic features at different scales to form a multi-scale feature map. Based on the multi-scale feature map, the system divides the image into multiple regional units, transforms them into graph nodes, and constructs a node feature set to establish spatial adjacency and feature similarity relationships between regions. By calculating the spatial distance and semantic similarity of each regional unit, the model generates an initial adjacency matrix and further adjusts the edge weight relationships based on an adaptive update strategy, combining spatial connectivity and semantic similarity, to form an adaptive adjacency matrix that reflects the true semantic structure of the image.

[0088] Building upon this foundation, the adaptive adjacency matrix and node feature set are input into an improved graph convolutional network for feature propagation and aggregation. The model performs feature extraction in parallel within both spatial and semantic graph convolution branches. A cross-attention module enables bidirectional interaction and weighted fusion of spatial and semantic features, achieving adaptive coordination of information across different feature channels at the semantic level. Simultaneously, a global attention fusion module performs weighted updates and recalculations on the node dimensions of the fused features, achieving redistribution of global context information and feature write-back. After multi-level aggregation and fusion, the node feature representation output by the model exhibits higher structural consistency and semantic completeness, significantly improving segmentation robustness and boundary accuracy in complex scenarios.

[0089] Finally, the system fuses node feature representations with multi-scale feature maps and performs upsampling and pixel-level mapping to restore region-level features into a semantic segmentation result map with the same size as the original image. In the underground parking lot scenario, the model can accurately segment vehicle outlines, ground passages, lane markings, pedestrians, and obstacles, and maintains high stability under varying lighting conditions and occlusion. Through continuous testing at different times and under different lighting conditions, the semantic segmentation results achieved by this invention show significant improvements in segmentation coherence, detail preservation, and category boundary accuracy in complex backgrounds.

[0090] This embodiment demonstrates that by introducing the construction of an adaptive adjacency matrix and a bidirectional attention fusion mechanism, the present invention significantly enhances the model's ability to understand complex spatial structures and semantic hierarchical relationships, making the image semantic segmentation results more refined and robust. It can stably adapt to image input under different environmental conditions, and improve the recognition accuracy and response efficiency of intelligent security systems for dynamic scenes.

[0091] Table 1. Performance Comparison of the Invention and Traditional Image Semantic Segmentation Methods

[0092]

[0093] As can be clearly seen from Table 1, the method of the present invention is superior to the traditional method in many indicators.

[0094] This invention achieves an average intersection-union ratio of 90.3%, which is 11.7 percentage points higher than FCN and 7.9 percentage points higher than DeepLabv3+. This demonstrates that this invention significantly outperforms traditional convolutional architectures in terms of overall region segmentation accuracy. The core reason for this is that this invention dynamically characterizes the spatial and semantic relationships between graph nodes by constructing an adaptive adjacency matrix, replacing the local convolution method with a fixed receptive field, enabling the model to more accurately capture cross-region structural dependencies.

[0095] In terms of pixel accuracy and class average accuracy, this invention achieves 97.6% and 93.1% respectively, representing improvements of 6.4% and 9.4% compared to FCN, and 4.1% and 6.9% compared to DeepLabv3+. This improvement stems from the joint modeling mechanism of spatial graph convolutional units and semantic graph convolutional units in this invention. Through the bidirectional interaction between spatial aggregation features and semantic aggregation features, the semantic offset problem in feature layering is effectively eliminated, enabling the model to more accurately distinguish semantically similar but spatially different regions.

[0096] In terms of boundary recognition performance, the present invention achieves a boundary F1 score of 89.5%, which is 13.6 percentage points higher than FCN and 9.2 percentage points higher than DeepLabv3+, significantly reducing boundary blurring and target adhesion problems. This result is attributed to the introduction of the cross-attention module, which establishes a dynamic response channel between spatial and semantic features through bidirectional weighted mapping operations. This allows the model to automatically strengthen salient features at the boundary during feature fusion, thereby achieving clearer segmentation boundaries.

[0097] In robustness tests under complex environments, the present invention achieved a low-light scene robustness score of 88.7% and an occluded scene recognition retention rate of 86.9%, representing improvements of 15.9% and 17.6% respectively compared to FCN, and improvements of 12.2% and 12.8% respectively compared to DeepLabv3+. This result fully demonstrates that the present invention can maintain stable segmentation performance under uncertain conditions such as environmental changes, uneven illumination, and local occlusion. This advantage stems from the global context modeling capability of the global attention fusion module in the present invention, which achieves adaptive enhancement of key semantic regions through global weighting and feature rewriting of node features.

[0098] Furthermore, in terms of model stability metrics, the result of this invention is 0.037, significantly lower than FCN's 0.082 and DeepLabv3+'s 0.065, indicating that the model exhibits small output fluctuations under multiple batches of input and different data conditions, demonstrating good generalization ability. This is attributed to the feature normalization and global context correction mechanisms of this invention, which effectively suppress gradient drift and semantic overfitting.

[0099] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An image semantic segmentation method based on graph convolutional networks, characterized in that, Includes the following steps: Obtain the raw image data, perform preprocessing, and obtain a standardized image tensor; Feature extraction is performed on the standardized image tensor to capture feature information at different spatial and semantic levels layer by layer, generating multi-scale feature maps. Region partitioning is performed based on multi-scale feature maps. The partitioned region units are represented as graph nodes, and a node feature vector is assigned to each graph node to construct a node feature set. Based on the spatial proximity and feature similarity relationships between regional units, the connection strength between regional units is calculated, and an initial adjacency matrix is ​​constructed. The initial adjacency matrix is ​​adaptively updated. The edge weight adjustment value is calculated based on the semantic similarity and spatial connectivity between node feature vectors. The adjacency relationship is corrected according to the adjustment result to generate an adaptive adjacency matrix. The adaptive adjacency matrix and the set of node features are input into the graph convolutional network, where graph convolution operations and feature aggregation are performed to output node feature representations enhanced by global context. The node feature representations and multi-scale feature maps are subjected to feature-level concatenation and weighted fusion to obtain fused features; The fused features are upsampled and mapped at the pixel level, and the region-level features are reconstructed to the original image resolution according to the spatial coordinates, generating a semantic segmentation result map that corresponds to the original image in terms of spatial size.

2. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The preprocessing includes image normalization, size normalization, channel alignment, noise suppression, data augmentation, and tensor quantization.

3. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The generation of the multi-scale feature map includes: performing multi-layer convolution processing on the standardized image tensor; using a sliding convolution kernel to perform weighted summation calculation on local regions within the spatial domain; extracting the local feature distribution under different receptive fields; sequentially performing nonlinear activation and batch normalization operations after the output of each convolutional layer; performing amplitude constraint and scale normalization processing on the feature response values; performing downsampling operations between adjacent convolutional layers; compressing the spatial dimension and retaining the main response region features through pooling operations; stacking the feature maps output from different levels of convolution in the channel dimension; performing feature fusion operations according to the hierarchical order; performing weighted superposition processing on high-level features and low-level features; and weighting and combining the channel responses of each layer's feature map during the fusion process, with the weight values ​​dynamically allocated according to the relative proportion of the feature response intensity of each layer, thereby generating a multi-scale feature map.

4. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The construction of the node feature set includes: performing spatial partitioning operations on the multi-scale feature map to divide the image into multiple regional units, each regional unit consisting of a spatially continuous set of pixels; extracting the corresponding feature response values ​​for each regional unit; calculating the feature mean and variance of all pixels within the regional unit on each channel; obtaining the statistical feature vector of the corresponding regional unit through weighted averaging; defining each regional unit as a graph node; assigning a unique node index to each graph node and establishing an index mapping relationship; extracting the center coordinate position parameters within the region corresponding to each graph node; performing channel concatenation and normalization processing with the statistical feature vector to generate a node feature vector; and arranging and aggregating all node feature vectors according to the node index order to obtain the node feature set.

5. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The construction of the initial adjacency matrix specifically includes: Based on the center coordinate position parameters and statistical feature vectors of each graph node in the node feature set, the spatial distance and feature similarity between any two graph nodes are calculated. The spatial distance is obtained by taking the square root of the sum of the squares of the differences in the node center coordinates, and the feature similarity is calculated by the ratio of the inner product and the product of the modulus between the node feature vectors. The spatial distance and feature similarity are normalized, the spatial distance is mapped to distance weight, and the feature similarity is mapped to similarity weight. The two types of weights are then weighted and combined according to a preset balance coefficient to generate a connection strength matrix. Valid connections between nodes are filtered based on the numerical threshold of the connection strength matrix. Connections below the threshold are assigned zero weights, while connections above the threshold retain their weights, thus obtaining preliminary adjacency relationships. Establish a matrix row and column mapping relationship based on the graph node index, fill the connection strength value of each node pair in the initial adjacency relationship into the corresponding matrix element position, fill the unconnected node pairs with zero values, and construct the initial adjacency matrix.

6. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The generation of the adaptive adjacency matrix specifically includes: Based on the initial adjacency matrix and node feature set, the similarity of the node feature vectors of each graph node is calculated to obtain the semantic similarity matrix. The similarity is obtained by calculating the cosine value between any two node feature vectors, where the cosine value is the ratio of the inner product of the two node feature vectors to the product of their respective magnitudes. Calculate the spatial distance between nodes based on the center coordinate position parameters of each node in the graph, and then normalize the distance by taking its reciprocal to obtain the spatial connectivity matrix. The semantic similarity matrix and the spatial connectivity matrix are weighted and fused according to the weighted fusion coefficients. The fusion result is then linearly normalized to generate an edge weight adjustment matrix. The weighted fusion coefficients are adaptively adjusted according to the node feature response intensity. The edge weights at corresponding positions in the initial adjacency matrix are corrected based on the element values ​​of the edge weight adjustment matrix. The difference matrix of the corrected edge weights is calculated, and the difference matrix is ​​weighted and superimposed to generate the updated adjacency relation matrix. The updated adjacency matrix is ​​normalized to maintain a stable numerical distribution, and a threshold pruning operation is performed on all edge weights to generate an adaptive adjacency matrix.

7. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The output of the node feature representation enhanced by global context specifically includes: An adaptive adjacency matrix and a set of node features are input into a graph convolutional network. Using the adaptive adjacency matrix as an update reference, the spatial adjacency matrix and the semantic adjacency matrix are parametrically learned and adjusted to generate a spatial association matrix and a semantic relation matrix. The graph convolutional network adopts a DGCN structure, and the spatial adjacency matrix and the semantic adjacency matrix in the original DGCN network are parametrically learned. A cross-attention module is added between the spatial graph convolutional unit and the semantic graph convolutional unit, and a global attention fusion module is added at the output of the dual branch. In the spatial graph convolutional unit, the feature set of nodes is aggregated based on the spatial correlation matrix. The features of the corresponding neighboring nodes are aggregated according to the spatial correlation weight for each graph node to generate spatial aggregated features. In the semantic graph convolutional unit, semantic relevance convolution processing is performed on the node feature set based on the semantic relation matrix. For each graph node, the features of the corresponding neighboring nodes are combined according to the semantic relevance weights to generate semantic aggregate features. Spatial aggregation features and semantic aggregation features are input into the cross-attention module, and the responses of the two types of feature channels are weighted and fused to generate a fused feature representation. The cross-attention module includes a feature alignment layer, a bidirectional attention weight calculation layer and a feature coupling layer. The fused feature representation is input into the global attention fusion module, which performs global weighted updates and recalculations on the features of the fused feature representation at the node dimension, and outputs a node feature representation enhanced by global context. The global attention fusion module includes a global feature statistics layer, a global weight allocation layer, and a feature write-back layer.

8. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The fusion feature is obtained by: aligning the node feature representation with the multi-scale feature map in the channel dimension; normalizing the size of each scale feature in the multi-scale feature map; concatenating the aligned multi-scale features with the node feature representation in the channel; weighting each channel feature in the concatenated feature map according to a preset fusion weight; calculating the weighted channel response value at each position; performing standardization and linear transformation on the weighted channel response; constraining the feature amplitude range; and rearranging and stacking the processed feature maps according to the node index order to obtain the fusion feature.

9. The image semantic segmentation method based on graph convolutional networks according to claim 1, characterized in that, The generation of the semantic segmentation result map includes: performing an upsampling operation on the fused features, expanding the feature map spatial size to the same resolution as the original image by a preset multiple, performing interpolation calculation and convolution smoothing during the upsampling process to supplement spatial details, performing pixel-level mapping on the upsampled feature mapping, converting the channel response values ​​into semantic category probability distributions, taking the category index value corresponding to the maximum probability of the semantic category probability distribution, generating a semantic label mapping matrix, adjusting the spatial arrangement of the semantic label mapping matrix to be consistent with the coordinate order of the original image, and generating the semantic segmentation result map.

Images