Explanatable semantic segmentation method based on superpixels and graph neural network

By constructing a dynamic graph structure based on superpixels and graph neural networks, interpretable segmentation feedback is enhanced, and pixel features are aggregated layer by layer. This solves the limitations of traditional semantic segmentation methods in capturing the semantic structure and spatial relationships of images, improves the transparency and accuracy of segmentation, and adapts to complex scenarios.

CN118942087APending Publication Date: 2024-11-12NANKAI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410985347.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional semantic segmentation methods have limitations in capturing the semantic structure and spatial relationships of images, lack transparency and interpretability, and are inflexible in handling complex scenes and changes.

Method used

We employ a superpixel and graph neural network-based approach, constructing a dynamic graph structure through multi-level fusion of superpixel generation and optimization techniques. This enhances the interpretable segmentation feedback mechanism, visualizes the pixel aggregation process layer by layer, and uses graph convolution and pooling blocks for semantic segmentation.

Benefits of technology

It improves the accuracy and interpretability of semantic segmentation, enhances the transparency of the model, facilitates the understanding and verification of segmentation results, and improves the adaptability and segmentation accuracy in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942087A_ABST
    Figure CN118942087A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of computer data processing, and particularly relates to an interpretable semantic segmentation method based on superpixels and a graph neural network. According to the method, the problem of a model black box is solved through an explanatory enhanced segmentation feedback mechanism, the segmentation flexibility and accuracy are improved by using a dynamic graph structure construction and adaptive topology updating technology, and on the basis, details and boundary information of an image are kept through a multi-level fused superpixel generation and optimization technology, so that the segmentation accuracy is improved. Therefore, the segmentation precision is obviously improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer data processing, and more specifically, relates to an interpretable semantic segmentation method based on superpixels and graph neural networks. Background Art

[0002] Semantic segmentation is one of the most basic and critical tasks in the field of computer vision and an important component of the underlying perception module in applications such as autonomous driving. Its core task is to classify each pixel in the image, accurately locate the area where the object is located, and eliminate background interference.

[0003] With the rapid development of deep learning technology, semantic segmentation technology based on convolutional neural network (CNN) has made significant progress. In particular, the emergence of methods such as fully convolutional network (FCN) has greatly improved the accuracy and efficiency of segmentation. However, the traditional pixel-level CNN model still has certain limitations in capturing the spatial relationship between objects in the image. To solve this problem, graph neural network (GNN) was introduced, which can process non-Euclidean structured data and show superior performance in various image processing tasks.

[0004] GNN effectively improves the accuracy and robustness of segmentation by extracting the topological structure features of the image and using the global context for reasoning. However, most GNN-based methods still directly predict the final category of the pixel, lacking an intermediate aggregation process, which makes it difficult to explain the errors in the segmentation process. Superpixels, as a pixel aggregation method, can explain the spatial relationship between pixels to a certain extent, but existing methods usually do not directly generate semantic segmentation results. Summary of the invention

[0005] To solve such problems, the present invention proposes an interpretable semantic segmentation method based on superpixels and graph neural networks, thereby effectively improving the accuracy and interpretability of semantic segmentation. The present invention solves the following technical problems:

[0006] 1. In response to the problem of insufficient transparency and interpretability of the semantic segmentation process, the present invention proposes a segmentation feedback mechanism with enhanced interpretability to achieve real-time monitoring and adjustment of errors in the segmentation process, thereby enhancing the transparency and interpretability of the model and facilitating the understanding and verification of the segmentation results.

[0007] 2. In view of the fact that traditional semantic segmentation methods are difficult to accurately capture the semantic structure of images, the present invention uses multi-level fusion superpixel generation and optimization technology to accurately capture the semantic structure of images through primary generation and optimization adjustment, enhance the spatial relationship representation between pixels, and maintain the details and boundary information of the image, significantly improving the segmentation accuracy of pixel areas.

[0008] 3. In view of the lack of flexibility and accuracy of traditional methods in dealing with complex scenes and changes, the present invention introduces dynamic graph structure construction and adaptive topology update, builds and adjusts the graph structure in real time, enables the graph neural network to adapt to complex scenes and changes, and improves the flexibility and accuracy of segmentation.

[0009] Aiming at the traditional pixel-level semantic segmentation method, the present invention proposes a segmentation feedback mechanism with enhanced interpretability. By visualizing the pixel aggregation process layer by layer, it can realize real-time detection and adjustment of errors in the segmentation process, enhance the transparency and interpretability of the model, and make the results easier to understand and adjust.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] An interpretable semantic segmentation method based on superpixels and graph neural networks includes the following steps:

[0012] Step 1: Input the original image into the superpixel segmentation network to construct graph structure data;

[0013] Step 2: Build a multi-layer semantic segmentation model based on graph neural network;

[0014] Step 3: Model initialization, inputting graph structure data into the multi-layer semantic segmentation model;

[0015] Step 4: The graph convolution block updates the node features, and the prediction block predicts the node classification;

[0016] Step 5: The graph pooling block aggregates node information and builds a new graph structure;

[0017] Step 6: Repeat steps 4 and 5 until the final semantic segmentation result is obtained;

[0018] Step 7: Input the test set graph structure data into the model and output the multi-layer semantic segmentation results.

[0019] The technical solution is further optimized, and the step 1 comprises:

[0020] The image is input into the superpixel segmentation network, and the multi-scale segmentation of the image is performed using the multi-level fusion superpixel generation and optimization technology to obtain a multi-scale superpixel representation; each superpixel is regarded as a node, and while performing superpixel segmentation, the multi-dimensional information of each pixel is extracted and combined with the average color and position information of the original image to form rich superpixel features. The calculation formula is as follows:

[0021]

[0022] Among them, M j , C j , P jRespectively represent the multi-dimensional feature information, color information and position information of the jth superpixel, N j represents the number of pixels contained in the jth superpixel, S i represents the multidimensional feature information of the i-th pixel in the j-th superpixel, R i , G i , B i represents the color information of the i-th pixel in the j-th superpixel, x i ,y i Represents the position information of the i-th pixel in the j-th superpixel; determines the connection between nodes according to the spatial relationship of superpixels, and generates the graph structure data of the original image.

[0023] The technical solution is further optimized. The generation of the graph structure data includes: overlapping the superpixel segmentation result of the original image with the real label image, performing the same superpixel segmentation on the real label image, determining the label category of the superpixel according to the label category of most pixels in the same segmentation area, and obtaining the real label at the superpixel level. The calculation formula of the label category is as follows:

[0024]

[0025] Among them, L superpixel is the label of the superpixel, R is the set of pixels in the superpixel area, l i is the true label of pixel i, k is the possible label category, δ is the Kronecker function, for two variables i and j, the Kronecker function δ ij The definition is as follows:

[0026]

[0027] This means that when i and j are equal, δ ij is equal to 1, otherwise it is equal to 0;

[0028] In addition, the original image is downsampled multiple times, and the same superpixel segmentation is performed on the downsampled images. After the superpixel segmentation images of the downsampled images are restored to the original image size, the superpixel segmentation images of each downsampled image are overlapped with the superpixel images of the original image, and the correspondence between each superpixel of the original image and the superpixel of the downsampled image is established based on the overlapping area. The labels of the original superpixels are generated at all levels, and the cross entropy loss function is calculated using these labels to constrain the generation of the intermediate semantic layer. The calculation formula of the cross entropy loss function is as follows:

[0029]

[0030] Where C is the total number of categories, y i is the one-hot encoding vector of the label, the i-th element yi Indicates whether the sample belongs to the i-th category, with a value of 0 or 1, p i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1;

[0031] All the previously obtained information, including superpixel features, true labels, and down-sampling labels at all levels, are stored together as node features in the graph structure data.

[0032] The technical solution is further optimized, and the graph neural network model includes multiple graph convolution blocks, graph pooling blocks and prediction blocks, wherein each graph convolution block is composed of a graph convolution layer and an activation function, each graph pooling block is composed of a graph pooling layer and an activation function, and each prediction block is composed of a graph depooling layer, a fully connected layer and an activation function.

[0033] The technical solution is further optimized, and step three includes: inputting the processed graph structure data into the constructed graph neural network model for semantic segmentation training, obtaining the accuracy and intersection-over-union ratio of the model on the training set and the validation set, and selecting the model parameters with the highest intersection-over-union ratio on the validation set as the pre-training model.

[0034] The technical solution is further optimized, and step four includes: after the graph structure data is passed into the graph convolution block, the graph convolution layer updates the features of each node by aggregating the information of neighboring nodes; the features output by the graph convolution layer are nonlinearly transformed through the ReLU activation function, and then passed through the dropout layer to randomly discard some node features with a certain probability to prevent model overfitting; after the above operations, the new graph structure is restored to the original graph structure using the graph depooling layer, and then the node features are converted into category prediction vectors through the fully connected layer; the output of the fully connected layer is converted into a category probability distribution using the Softmax function, and the category corresponding to the maximum probability is selected as the predicted category of the node to obtain the semantic segmentation result.

[0035] The technical solution is further optimized, and step five includes: the graph pooling block aggregates nodes belonging to the same category according to the prediction results of the previous layer to simplify the graph structure, reduce the number of nodes and computational complexity; the features after pooling are merged with the features before pooling, and new node features are generated through weighted sum operation; the graph structure is reconstructed according to the new node features and spatial relationships, and the connection relationship between the nodes is updated; and the dynamic graph construction technology is used to adaptively adjust the topological structure of the graph, and the updated node features are passed to the next graph convolution block, and the process of graph convolution and graph pooling is repeated to extract and aggregate node features layer by layer until the semantic segmentation of all layers is completed.

[0036] The technical solution is further optimized, and the step six includes: using multiple layers of aggregated labels and real labels to perform multiple cross entropy loss function calculations to constrain the training process. The calculation formula of the loss function is as follows:

[0037]

[0038] Where C is the total number of categories, y i is the one-hot encoding vector of the label, the i-th element y i Indicates whether the sample belongs to the i-th category, and the value is 0 or 1; p i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1;

[0039] Using the intersection over union ratio as the verification indicator of the model, the IoU calculation formula is as follows:

[0040]

[0041] Among them, TP represents the correctly detected target area, FP represents the incorrectly detected target area, and FN represents the missed target area. After the training is completed, the graph neural network model with the best performance on the validation set is selected as the multi-layer semantic segmentation model.

[0042] Different from the prior art, the above technical solution has the following beneficial effects:

[0043] 1. The present invention proposes an interpretable semantic segmentation method based on superpixels and graph neural networks, which makes up for the shortcomings of traditional semantic segmentation methods in terms of interpretability, emphasizes the transparency of the segmentation process, increases people's trust in the semantic segmentation results, and helps to implement applications in fields such as autonomous driving.

[0044] 2. By using superpixel preprocessing technology, the image is divided into superpixel regions, which effectively maintains the details and boundary information of the image and improves the accuracy of the segmentation results.

[0045] 3. Use graph structured data and graph neural networks to process spatial relationships in images, improve the model's adaptability in complex scenarios, and enable it to better handle the spatial relationships between objects in images.

[0046] A layer-by-layer conversion process is adopted to show the process of aggregating the original image from pixels layer by layer into super-pixel images of different sizes, and finally generating a semantic segmentation result map, which enhances the interpretability of the segmentation process and facilitates the understanding and verification of the segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart for the implementation of an interpretable semantic segmentation method based on superpixels and graph neural networks;

[0048] Figure 2 This is a diagram of the structure of an interpretable semantic segmentation model based on superpixels and graph neural networks;

[0049] Figure 3 Schematic diagram of the explainable semantic segmentation method based on superpixels and graph neural networks. DETAILED DESCRIPTION

[0050] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0051] This paper proposes an interpretable semantic segmentation method based on superpixels and graph neural networks. It solves the model black box problem by enhancing the interpretable segmentation feedback mechanism, and uses dynamic graph structure construction and adaptive topology update technology to improve segmentation flexibility and accuracy. On this basis, the details and boundary information of the image are maintained through multi-level fusion superpixel generation and optimization technology, thereby significantly improving the segmentation accuracy.

[0052] The task is to use superpixels and graph neural networks to perform interpretable multi-layer semantic segmentation on images on a vehicle traffic dataset, intuitively display the semantic segmentation pixel aggregation process, improve segmentation accuracy, and increase the transparency of the semantic segmentation algorithm.

[0053] See also Figure 1 As shown, it is an implementation flow chart of the interpretable semantic segmentation method based on superpixels and graph neural networks, and the method includes the following steps:

[0054] Step 1: Input the original image into the superpixel segmentation network to construct graph structure data

[0055] The image is input into the superpixel segmentation network, and the multi-scale segmentation of the image is performed using the multi-level fusion superpixel generation and optimization technology to obtain a multi-scale superpixel representation. Each superpixel is regarded as a node. While performing superpixel segmentation, the multi-dimensional information of each pixel is extracted and combined with the average color and position information of the original image to form rich superpixel features. The calculation formula is as follows:

[0056]

[0057] Among them, M j , C j , P j Respectively represent the multi-dimensional feature information, color information and position information of the jth superpixel, N j represents the number of pixels contained in the jth superpixel, S i represents the multidimensional feature information of the i-th pixel in the j-th superpixel, R i , Gi , B i represents the color information of the i-th pixel in the j-th superpixel, x i ,y i Represents the position information of the i-th pixel in the j-th superpixel.

[0058] The connection between nodes is determined according to the spatial relationship of superpixels, the relationship between the edges of the original image is obtained and the graph structure is constructed.

[0059] Overlap the superpixel segmentation result of the original image with the real label image, and perform the same superpixel segmentation on the real label image. Determine the label category of the superpixel based on the label category of most pixels in the same segmentation area, and obtain the real label at the superpixel level. The calculation formula of the label category is as follows:

[0060]

[0061] Among them, L superpixel is the label of the superpixel, R is the set of pixels in the superpixel area, l i is the true label of pixel i, k is the possible label category, and δ is the Kronecker function. For two variables i and j, the Kronecker function δ ij The definition is as follows:

[0062]

[0063] This means that when i and j are equal, δ ij is equal to 1, otherwise it is equal to 0.

[0064] In addition, the original image is downsampled multiple times, and the same superpixel segmentation is performed on the downsampled images. After the superpixel segmentation images of the downsampled images are restored to the size of the original images, the superpixel segmentation images of each downsampled image are overlapped with the superpixel images of the original image, and the correspondence between each superpixel of the original image and the superpixel of the downsampled image is established based on the overlapping area. The labels of the original superpixels are generated at all levels, and the cross entropy loss function is calculated using these labels to constrain the intermediate semantic layer. The calculation formula of the cross entropy loss function is as follows:

[0065]

[0066] Where C is the total number of categories, y i is the one-hot encoding vector of the label, the i-th element y i Indicates whether the sample belongs to the i-th category, and the value is 0 or 1. i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1.

[0067] All the previously obtained information, including superpixel features, true labels, and downsampling labels at all levels, are stored together as node features in the graph structure data to prepare for subsequent model training.

[0068] Step 2: Build a multi-layer semantic segmentation model based on graph neural network

[0069] Reference Figure 2 , is a structural diagram of an interpretable semantic segmentation model based on superpixels and graph neural networks, and a multi-layer semantic segmentation model based on graph neural networks is built. The graph neural network model includes multiple graph convolution blocks, graph pooling blocks and prediction blocks. Each graph convolution block consists of a graph convolution layer and an activation function, each graph pooling block consists of a graph pooling layer and an activation function, and each prediction block consists of a graph de-pooling layer, a fully connected layer and an activation function. After the input data is passed into the first graph convolution block, the node features are updated through the graph convolution layer, and a nonlinear transformation is performed through the activation function. Then, the node features output by the graph convolution block are passed into the prediction block, and preliminary node classification is performed through the graph de-pooling layer, the fully connected layer and the activation function to obtain the first layer of semantic segmentation image. Next, the node features output by the graph convolution block are passed into the first graph pooling block, and the node information is aggregated through the graph pooling layer to generate a new graph structure, and then a nonlinear transformation is performed through the activation function. In the above way, the process of graph convolution block, prediction block and graph pooling block is repeated many times, each layer outputs the corresponding semantic segmentation image, and updates the node features layer by layer to build a new graph structure until it is passed to the last graph pooling block. Finally, the node features output by the last graph pooling block are passed to the prediction block, and the original graph structure size is restored through the graph depooling layer. The fully connected layer performs linear transformation on the node features, and the activation function performs nonlinear transformation to obtain the final node classification result. The node classification results of all levels are combined to obtain the multi-layer semantic segmentation result of the entire graph.

[0070] This multi-level semantic segmentation model uses multiple graph convolutions and graph pooling processes to enable node features to be transmitted and aggregated layer by layer in the graph structure, achieving efficient semantic segmentation of complex image data and outputting accurate multi-layer semantic segmentation images.

[0071] Step 3: Model initialization, inputting graph structure data into the multi-layer semantic segmentation model

[0072] Organize and store the processed graph structure data of the training set and validation set separately. Use a custom data loader to batch load the graph structure data into the built graph neural network model. Initialize the parameters of each layer of the graph neural network model, including the weights and biases of the graph convolution layer, graph pooling layer, and fully connected layer. Use appropriate initialization methods to ensure the stability and convergence speed of the model. After each training cycle, use the data of the validation set for evaluation, and calculate evaluation indicators such as the accuracy and intersection-over-union ratio of the validation set. Select the model parameters with the highest intersection-over-union ratio on the validation set and save it as a pre-trained model for use in subsequent steps.

[0073] Step 4: The graph convolution block updates the node features, and the prediction block predicts the node classification

[0074] After the graph structure data is passed into the graph convolution block, the graph convolution layer updates the features of each node by aggregating the information of neighboring nodes. The features output by the graph convolution layer are transformed nonlinearly through the ReLU activation function, and then passed through the dropout layer to randomly discard some node features with a certain probability to prevent the model from overfitting; after the above operations, the new graph structure is restored to the original graph structure using the graph depooling layer, and then the node features are converted into category prediction vectors through the fully connected layer; the output of the fully connected layer is converted into a category probability distribution using the Softmax function, and the category corresponding to the maximum probability is selected as the predicted category of the node to obtain the semantic segmentation result.

[0075] Step 5: Graph pooling blocks aggregate node information and build a new graph structure

[0076] The graph pooling block aggregates nodes belonging to the same category according to the prediction results of the previous layer to simplify the graph structure, reduce the number of nodes and computational complexity. The pooled features are fused with the features before pooling, and new node features are generated through weighted sum operations. According to the new node features and spatial relationships, the graph structure is rebuilt and the connection relationship between nodes is updated. The dynamic graph construction technology is used to adaptively adjust the topological structure of the graph. The updated node features are passed to the next graph convolution block, and the graph convolution and graph pooling process are repeated to extract and aggregate node features layer by layer until the semantic segmentation of all layers is completed.

[0077] Step 6: Repeat steps 4 and 5 until the final semantic segmentation result is obtained

[0078] Multiple cross entropy loss function calculations are performed using multiple layers of aggregated labels and true labels to constrain the training process. The loss function calculation formula is as follows:

[0079]

[0080] Where C is the total number of categories, y iis the one-hot encoding vector of the label, the i-th element y i Indicates whether the sample belongs to the i-th category, and the value is 0 or 1. i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1.

[0081] The intersection over union (IoU) is used as the verification indicator of the model. The IoU calculation formula is as follows:

[0082]

[0083] Among them, TP represents the correctly detected target area, FP represents the incorrectly detected target area, and FN represents the missed target area.

[0084] The trained model is applied to the validation set data, and hyperparameters are optimized by adjusting the model's learning rate, batch size, number of iterations, and other parameters to improve the accuracy of the validation set and evaluation indicators such as intersection-over-union. Feedback and adjustments are made to the model through these evaluation indicators to ensure that the optimization direction of the model is correct. After training, the graph neural network model with the best performance on the validation set is selected as the multi-layer semantic segmentation model for this time.

[0085] Step 7: Input the test set graph structure data into the model and output the multi-layer semantic segmentation results

[0086] The test set images that need semantic segmentation are preprocessed through step 1, and then input into the graph neural network model obtained in step 6 to complete the multi-layer semantic segmentation of the images. Figure 3 , which intuitively shows the whole process. When the input image passes through the graph neural network model, multiple prediction matrices Q1, Q2, ..., Qn are generated. After these prediction matrices are processed by the Softmax layer, semantic segmentation images 1, 2, ..., n are generated respectively, and multi-layer semantic segmentation images are obtained by combining them.

[0087] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "include..." or "comprise..." do not exclude the existence of other elements in the process, method, article or terminal device including the elements. In addition, in this article, "greater than", "less than", "exceed" and the like are understood to exclude the number itself; "above", "below", "within" and the like are understood to include the number itself.

[0088] Although the above embodiments have been described, once those skilled in the art know the basic creative concepts, they can make additional changes and modifications to these embodiments. Therefore, the above description is only an embodiment of the present invention and does not limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An interpretable semantic segmentation method based on superpixels and graph neural networks, characterized in that: The following steps are involved: Step 1: Input the original image into the superpixel segmentation network to construct graph structure data; Step 2: Build a multi-layer semantic segmentation model based on graph neural network; Step 3: Model initialization, inputting graph structure data into the multi-layer semantic segmentation model; Step 4: The graph convolution block updates the node features, and the prediction block predicts the node classification; Step 5: The graph pooling block aggregates node information and builds a new graph structure; Step 6: Repeat steps 4 and 5 until the final semantic segmentation result is obtained; Step 7: Input the test set graph structure data into the model and output the multi-layer semantic segmentation results.

2. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The step one comprises: Input the image into the superpixel segmentation network, use the multi-level fusion superpixel generation and optimization technology to perform multi-scale segmentation on the image and obtain multi-scale superpixel representation; Considering each superpixel as a node, while performing superpixel segmentation, the multi-dimensional information of each pixel is extracted and combined with the average color and position information of the original image to form rich superpixel features. The calculation formula is as follows: Among them, M j , C j , P j Respectively represent the multi-dimensional feature information, color information and position information of the jth superpixel, N j represents the number of pixels contained in the jth superpixel, S i represents the multidimensional feature information of the i-th pixel in the j-th superpixel, R i , G i , B i represents the color information of the i-th pixel in the j-th superpixel, x i ,y i Represents the position information of the i-th pixel in the j-th superpixel; determines the connection between nodes according to the spatial relationship of superpixels, and generates the graph structure data of the original image.

3. The interpretable semantic segmentation method based on superpixels and graph neural networks as claimed in claim 2, characterized in that: The generation of the graph structure data includes: overlapping the superpixel segmentation result of the original image with the real label image, performing the same superpixel segmentation on the real label image, determining the label category of the superpixel according to the label category of most pixels in the same segmentation area, and obtaining the real label at the superpixel level. The calculation formula of the label category is as follows: Among them, L superpixel is the label of the superpixel, R is the set of pixels in the superpixel area, l i is the true label of pixel i, k is the possible label category, δ is the Kronecker function, for two variables i and j, the Kronecker function δ ij The definition is as follows: This means that when i and j are equal, δ ij is equal to 1, otherwise it is equal to 0; In addition, the original image is downsampled multiple times, and the same superpixel segmentation is performed on the downsampled images. After the superpixel segmentation images of the downsampled images are restored to the original image size, the superpixel segmentation images of each downsampled image are overlapped with the superpixel images of the original image. The correspondence between each superpixel of the original image and the superpixel of the downsampled image is established based on the overlapping area, and labels of various levels of the original superpixels are generated. These labels are used to calculate the cross entropy loss function, thereby constraining the generation of the intermediate semantic layer. The calculation formula of the cross entropy loss function is as follows: Where C is the total number of categories, y i is the one-hot encoding vector of the label, the i-th element y i Indicates whether the sample belongs to the i-th category, with a value of 0 or 1, p i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1; All the previously obtained information, including superpixel features, true labels, and down-sampling labels at all levels, are stored together as node features in the graph structure data.

4. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The graph neural network model includes multiple graph convolution blocks, graph pooling blocks and prediction blocks, wherein each graph convolution block consists of a graph convolution layer and an activation function, each graph pooling block consists of a graph pooling layer and an activation function, and each prediction block consists of a graph depooling layer, a fully connected layer and an activation function.

5. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The step three includes: inputting the processed graph structure data into the constructed graph neural network model for semantic segmentation training, obtaining the accuracy and intersection-over-union ratio of the model on the training set and the validation set, and selecting the model parameters with the highest intersection-over-union ratio on the validation set as the pre-training model.

6. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The step four includes: after the graph structure data is passed into the graph convolution block, the graph convolution layer updates the features of each node by aggregating the information of neighboring nodes; the features output by the graph convolution layer are nonlinearly transformed through the ReLU activation function, and then passed through the dropout layer to randomly discard some node features with a certain probability to prevent the model from overfitting; after the above operations, the new graph structure is restored to the original graph structure using the graph depooling layer, and then the node features are converted into category prediction vectors through the fully connected layer; the output of the fully connected layer is converted into a category probability distribution using the Softmax function, and the category corresponding to the maximum probability is selected as the predicted category of the node to obtain the semantic segmentation result.

7. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The step five includes: the graph pooling block aggregates nodes belonging to the same category according to the prediction results of the previous layer to simplify the graph structure, reduce the number of nodes and computational complexity; the features after pooling are merged with the features before pooling, and new node features are generated through weighted sum operation; the graph structure is rebuilt according to the new node features and spatial relationships, and the connection relationship between nodes is updated; and the dynamic graph construction technology is used to adaptively adjust the topological structure of the graph, and the updated node features are passed to the next graph convolution block, and the process of graph convolution and graph pooling is repeated to extract and aggregate node features layer by layer until the semantic segmentation of all layers is completed.

8. The interpretable semantic segmentation method based on superpixels and graph neural networks according to claim 1, characterized in that: The step six includes: using the multi-layer aggregated labels and the real labels to perform multiple cross entropy loss function calculations to constrain the training process. The calculation formula of the loss function is as follows: Where C is the total number of categories, y i is the one-hot encoding vector of the label, the i-th element y i Indicates whether the sample belongs to the i-th category, and the value is 0 or 1; p i The probability that the sample predicted by the model belongs to the i-th category ranges from 0 to 1, and the sum of the probabilities of all categories is 1; Using the intersection over union ratio as the verification indicator of the model, the IoU calculation formula is as follows: Among them, TP represents the correctly detected target area, FP represents the incorrectly detected target area, and FN represents the missed target area. After the training is completed, the graph neural network model with the best performance on the validation set is selected as the multi-layer semantic segmentation model for this time.