A substation equipment leaked oil segmentation method based on double graph reasoning

Abstract

The application provides a substation equipment leakage oil segmentation method based on double graph reasoning, which comprises the following steps: constructing a data set, taking manually collected substation inspection pictures as a data source, selecting substation inspection images containing leakage oil defects to construct a data set, labeling the data set by using a labeling software, dividing a training set and a test set, building a leakage oil segmentation model, completing iterative training of the model based on the training set, and calculating the visualization and evaluation indexes of the segmentation effect of the model in the test set. The substation equipment leakage oil segmentation method based on double graph reasoning provided by the application utilizes the texture and edge characteristics of the leakage oil, and solves the problems of difficult defect feature extraction, low segmentation precision and poor segmentation effect caused by the strong correlation of the leakage oil components, the variable morphology and the diffusion change characteristics.

Description

Technical Field

[0001] This invention relates to the field of power technology, and in particular to a method for segmenting leaking oil in substations based on dual-graph reasoning. Background Technology

[0002] With the accelerated construction of new power systems, substation operation and maintenance modes are transitioning from unmanned to intelligent. Centralized processing and analysis of images from existing substation monitoring systems using deep learning methods such as object detection and semantic segmentation can significantly shorten inspection cycles and improve efficiency. For oil leaks in substation equipment, which are typically characterized by irregular and weak semantic defects, using rectangular bounding boxes for annotation can cause semantic ambiguity, interfering with the training of object detection algorithms and leading to poor detection results. Therefore, semantic segmentation algorithms are more suitable for oil leak detection tasks.

[0003] Oil leakage, as a weakly edged and irregular surface defect in substation equipment, is characterized by strong component correlation, variable morphology, and indistinct edges, significantly increasing the difficulty of segmentation. First, small-area oil leakage often occurs at the connection points of oil-filled substation components, while large-area leakage often spans multiple components. The strong component correlation makes feature extraction networks more likely to focus on component features rather than defect features. Second, the color and shape of the oil leakage are dynamically changing, making it difficult to extract common defect features at the shallow level of the feature extraction network. Furthermore, oxidized oil leakage areas and shadows share certain similarities in color, shape, and other shallow features. The semantic segmentation encoding and decoding mechanism directly introduces shallow features, easily leading to missed detections of shadowed oil leakage areas and false detections of shadowed background areas. Finally, oil leakage exhibits a point-to-surface diffusion characteristic, with leakage areas showing clear inner edges and blurred outer edges. The upsampling and downsampling mechanisms of semantic segmentation further weaken edge information, easily resulting in coarse segmentation edges. Therefore, designing a substation oil leakage segmentation method based on dual-graph reasoning is essential. Summary of the Invention

[0004] The purpose of this invention is to provide a method for segmenting oil leakage in power equipment based on dual-graph reasoning. This method utilizes the texture and edge characteristics of the oil leakage to solve the problems of difficulty in extracting defect features, low segmentation accuracy, and poor segmentation effect caused by the strong correlation, variable morphology, and diffusion characteristics of oil leakage components.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for segmenting oil leakage in power equipment based on dual-graph reasoning includes the following steps:

[0007] Step 1: Construct a dataset. Using manually collected images of substation equipment inspections as the data source, select images of substation equipment inspections containing oil leakage defects to construct a dataset. Use annotation software to annotate the dataset and divide it into training and test sets.

[0008] Step 2: Build an oil leakage segmentation model and complete iterative training of the model based on the training set;

[0009] Step 3: Use the test set to visualize the model segmentation effect and calculate the evaluation metrics.

[0010] Optionally, in step 1, a dataset is constructed using manually collected substation equipment inspection images as the data source. Images of substation equipment with oil leakage defects are selected to construct the dataset. The dataset is then labeled using annotation software, and divided into training and testing sets, specifically:

[0011] A dataset was constructed using manually collected images of substation equipment inspections as the data source. Images of substation equipment inspections containing oil leakage defects were selected, and the image resolution was unified to 2048×1024. Oil leakage areas in the substation equipment inspection images were manually annotated using an annotation tool to obtain segmented and annotated images. Edge-annotated images were automatically generated using the Canny edge detection method. The original substation equipment inspection images, the corresponding segmented and annotated images, and the edge-annotated images were combined to form image samples. All image samples were integrated to construct a dataset, which was then divided into training and test sets in a 7:3 ratio.

[0012] Optionally, in step 2, iterative training of the model is completed based on the training set, specifically including the following steps:

[0013] Step 201: Extract multi-scale feature maps, including deep feature maps and shallow feature maps, through the feature extraction network of the oil leakage segmentation model. The deep feature maps are rich in semantic information, while the shallow feature maps are rich in spatial information.

[0014] Step 202: Concatenate and fuse the first two shallow feature maps. Use two-dimensional discrete wavelet transform to decompose the shallow feature map into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture. Concatenate and fuse the two sub-maps to obtain a structured texture feature map.

[0015] Step 203: Aggregate the texture information of the structured texture feature map by weighted average pooling. The texture information at the region level is represented by the feature vector of a single node, realizing the transformation from the structured texture feature map in the pixel space to the semantic node in the interaction space. By calculating the similarity coefficient between semantic nodes, a weighted adjacency matrix for information propagation between nodes is generated to complete the semantic graph representation. The node information aggregation and state update are realized by a two-layer graph convolutional neural network. After completing the semantic relationship representation and reasoning between nodes, the node matrix in the sparse space is mapped back to the pixel space to obtain the enhanced texture feature map.

[0016] Step 204: The high-resolution feature map aggregates spatial information at different scales through the Bottle2neck layer to obtain a node matrix. Under the guidance of deep semantic features, the similarity coefficient between node feature vectors is calculated to obtain a weighted adjacency matrix, thus completing the spatial graph construction. The node information aggregation and state update are completed using the simplest graph convolution. After dimensional transformation, the enhanced edge feature map is obtained.

[0017] Step 205: Fuse the enhanced texture feature map with the shallow feature map extracted by the feature extraction network, fuse the enhanced edge feature map with the deep feature map extracted by the feature extraction network, and obtain the segmentation prediction map through the semantic segmentation head;

[0018] Step 206: Calculate the segmentation loss and edge perception loss using the segmentation prediction map, edge-aware map, segmentation labeled image, and edge-aware image. Use an alternating parameter update strategy to alternately update the relevant weight coefficients during forward and backward propagation to complete the overall optimization of the model and complete the training.

[0019] Optionally, in step 201, multi-scale feature maps are extracted using the feature extraction network of the leaked oil segmentation model, including deep feature maps and shallow feature maps. The deep feature maps are rich in semantic information, while the shallow feature maps are rich in spatial information. Specifically:

[0020] Obtain the image to be segmented x, and use a ResNet50 network with dilated convolutions to extract multi-scale feature maps {x0,x1,x2,x3,x4} from the image to be segmented x. Here, {x0,x1} are high-resolution shallow feature maps, rich in spatial information, and {x2,x3,x4} are low-resolution deep feature maps, rich in semantic information for scene understanding. x0 is taken as the output feature map of the first three 3×3 convolutions of the ResNet50 network.

[0021] Optionally, in step 202, the first two shallow feature maps are spliced ​​and fused. A two-dimensional discrete wavelet transform is used to decompose the shallow feature map into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture. The two sub-maps are then spliced ​​and fused to obtain a structured texture feature map, specifically:

[0022] The shallow feature map x1 is upsampled, concatenated with x0, and then compressed through a 1×1 convolution to obtain a high-resolution feature map. for:

[0023]

[0024] In the formula, f1(·,w1) is a 1×1 convolution operation with compressed channel dimensions, w1 is the learnable weight parameter in the 1×1 convolution, and (·) up This indicates an upsampling operation implemented by bilinear interpolation. This represents the feature map concatenation operation, which uses two-dimensional discrete wavelet transform to... spa Decomposed into low-frequency feature sub-maps reflecting smooth texture and high-frequency feature sub-maps reflecting abrupt texture changes and for:

[0025]

[0026] In the formula, These represent low-frequency feature sub-images, horizontal high-frequency feature sub-images, vertical high-frequency feature sub-images, and diagonal high-frequency feature sub-images, respectively. 2DWT represents the two-dimensional wavelet transform operation, and the selected wavelet basis function is Daubechies. The four sub-images are fused and concatenated, and the channel dimension is compressed through a 1×1 convolution to obtain the texture feature map. for:

[0027]

[0028] In the formula, f2(·,w2) is a 1×1 convolution operation that compresses the channel dimension, and w2 is the learnable weight parameter in the 1×1 convolution. This indicates a feature map splicing operation.

[0029] Optionally, in step 203, the texture information of the structured texture feature map is aggregated by weighted average pooling. The texture information at the region level is represented by a single node feature vector, realizing the transformation from the structured texture feature map in the pixel space to semantic nodes in the interaction space. By calculating the similarity coefficient between semantic nodes, a weighted adjacency matrix for information propagation between nodes is generated, completing the semantic graph representation. Node information aggregation and state update are realized through a two-layer graph convolutional neural network. After completing the semantic relationship representation and reasoning between nodes, the node matrix in the sparse space is mapped back to the pixel space to obtain the enhanced texture feature map. Specifically:

[0030] Semantic graph reasoning is achieved through the semantic graph convolution module of the oil leakage segmentation model. First, a projection function Ω(·) is constructed to transform the structured texture feature map in the pixel space. Node matrix X mapped to the interaction spacet ∈R N×N To achieve the aggregation of regional texture features, the projection function is represented as weighted average pooling:

[0031]

[0032] In the formula, This represents a set of learnable weight vectors. v represents the texture feature vector decomposed according to spatial dimensions. i ∈R N×1 The aggregated feature vector is represented by the projection function modeled as two convolution operations. With θ(·,w θ ), Used for dimensional transformation, θ(·,w) θ ) is used to generate the weight vector b i The calculation of the node matrix is ​​simplified to:

[0033]

[0034] In the formula, This represents matrix multiplication. w θ Let be the learnable parameters in the two convolutional layers, (·) vec This indicates a matrix redirection operation, (·). T This represents the transpose operation. After mapping, modeling the semantic adjacency relationships between texture features in any region of the structured texture feature map is simplified to capturing the node matrix X. t The adjacency relationship between the feature vectors of the nodes is determined by two one-dimensional convolution operations φ1(·,w). φ1 ) and φ2(·,w φ2 The similarity coefficient between the feature vectors of captured nodes is used to measure the adjacency relationship between semantic nodes, resulting in a weighted adjacency matrix A. t ∈R N×N for:

[0035]

[0036] In the formula, w represents matrix multiplication. φ1 ,w φ2 Let be the learnable parameters in two one-dimensional convolutions, (·) T The transpose operation is represented. After constructing the node matrix and adjacency matrix, semantic graph node information aggregation and state update are implemented through a two-layer graph convolutional neural network. During training, the adjacency matrix learns the edge weights of node features, which reflect the global relationships of structured texture features in the dense pixel space. After information diffusion, each node is aggregated to all necessary node information, and its state is updated through a linear transformation, resulting in:

[0037]

[0038] In the formula, This represents the node matrix after the state update, and ReLU represents the ReLU activation operation. This represents the adjacency matrix smoothed using the degree matrix. To represent matrix multiplication, Laplace smoothing is replaced by the softmax activation operation. W represents the node matrix before the state update. (0) With W (1) Given the learnable weight matrices in the two GCN layers, after semantic graph inference is completed, a back-projection function is constructed. Following the approach of feature aggregation, the back projection function is also modeled as two convolution operations. With θ′(·,w θ′ ),in Used for dimension reduction, θ′(·,w θ′ Used for feature diffusion to enhance texture features. Characterized as:

[0039]

[0040] Simplify the calculation, θ′(x tex ,w θ′ Directly take θ(x) tex ,w θ The transpose of ) To restore the learnable parameters in the convolutional layer to the required dimensions, (·) maj This indicates a vector-to-matrix conversion operation.

[0041] Optionally, in step 204, the high-resolution feature map aggregates spatial information at different scales through a Bottle2neck layer to obtain a node matrix. Guided by deep semantic features, the similarity coefficient between node feature vectors is calculated to obtain a weighted adjacency matrix, completing the spatial graph construction. Then, the node information aggregation and state update are completed using minimal graph convolution. After dimensionality transformation, an enhanced edge feature map is obtained, specifically:

[0042] Spatial graph inference is achieved through multiple cascaded spatial graph convolutional modules in the oil leakage segmentation model. Let t∈1,2,...,m be the positional encoding of the spatial graph convolution. Taking a single spatial graph inference module as an example, its input is a shallow feature map. With deep feature maps in Take the output feature map of the (t-1)th spatial map convolution to provide local contour detail features. When t=1, Take high-resolution feature map x spa, Sequentially extract the deep output feature maps x2, x3, and x4 from the backbone network, compress the channel dimension of both the deep and shallow feature maps to 1 using 1×1 convolutions, and then align the deep feature maps using upsampling operations. With shallow feature map The spatial dimensions are merged, spliced, and flattened to obtain a fused feature map. for:

[0043]

[0044] In the formula, f2(·,w2) and f3(·,w3) are 1×1 convolution operations that compress the channel dimension, w2 and w3 are the learnable weight parameters in the 1×1 convolution, and (·) up This indicates an upsampling operation implemented by bilinear interpolation. This indicates a feature map concatenation operation, (·) vec Indicates matrix steering operations to fuse feature maps. As input, the input feature map is processed using global adaptive average pooling. The spatial dimension is compressed to 1, and the weight coefficients between deep and shallow features are obtained through the sigmoid activation function. After diagonalization, a diagonal matrix Λ∈R is obtained. 2×2 ,for:

[0045]

[0046] In the formula, Sigmoid represents the sigmoid activation operation. express The feature vector obtained after global adaptive average pooling, where diag represents the diagonalization operation, is transformed using two 1×1 convolution-based transformation functions from the input feature map. Transform into and Perform a transpose operation on Q, calculate the similarity matrix using matrix multiplication, and then activate it using the softmax function to obtain the node feature vector x. i With node feature vector x j Normalized weights of spatial adjacency Calculate the normalized weighted adjacency matrix for:

[0047]

[0048] In the formula, Softmax represents the softmax activation operation. w represents matrix multiplication. φ3 ,w φ4 The parameters that can be learned in two one-dimensional convolutions, the node matrix Then from the shallow feature map After extracting multi-scale features through the Bottle2neck layer, the results are obtained by dimensionality transformation and transpose operation, i.e.:

[0049]

[0050] In the formula, μ(·,w μ ) indicates the use of the Bottle2neck layer, w μ This represents the learnable weight parameters in the Bottle2neck layer, (·). vec This represents a matrix-to-vector operation. After constructing the node matrix and adjacency matrix, it aggregates node information and updates the state of the spatial graph through a simplified graph convolution. A Laplacian smoothing operation is performed on the adjacency matrix to propagate node features across the spatial graph. After aggregating all necessary node information, the node state is updated using a learnable weight matrix, and enhanced edge features are obtained through dimensionality transformation. for:

[0051]

[0052] In the formula, ReLU represents the activation operation using ReLU, and D e It is the adjacency matrix A e The degree matrix W is used to perform the Laplace smoothing operation. e Learnable weight matrix Represents matrix multiplication, (·) mat The vector-to-matrix operation simplifies the calculation of enhanced edge features, as follows:

[0053]

[0054] x spa Guided by deep semantic features x2, x3, and x4, three consecutive spatial graph convolutional modules are used to enable the system to perceive the details of the oil leakage edge from the perspective of spatial adjacency, ultimately resulting in enhanced edge features. After further compressing the channel dimension through 1×1 convolution and activating with the Sigmoid function, the edge-aware map E∈R is obtained. 1×H×W Simultaneously, 1×1 convolution is used to expand the channel dimension to obtain an enhanced edge feature map to supplement spatial information. for:

[0055]

[0056]

[0057] In the formula, f4(·,w4) is a 1×1 convolution operation with compressed channel dimensions, w4 is the learnable weight parameter in the 1×1 convolution, Sigmiod indicates activation operation using the sigmiod function, f5(·,w5) is a 1×1 convolution operation with compressed channel dimensions, and w5 is the learnable weight parameter in the 1×1 convolution.

[0058] Optionally, in step 205, the enhanced texture feature map is fused with the shallow feature map extracted by the feature extraction network, and the enhanced edge feature map is fused with the deep feature map extracted by the feature extraction network. A segmentation prediction map is then obtained through the semantic segmentation head. Specifically:

[0059] Enhance texture feature map T graph The shallow feature map x1 extracted by the feature extraction network is fused together, and the edge feature map E is enhanced. s Compared with the deep feature map x after ASPP module aspp The feature maps are fused, and the fused shallow and deep feature maps are fed into the segmentation head to obtain the segmentation prediction map S∈R. 1×H×W .

[0060] Optionally, in step 206, the segmentation loss and edge-aware loss are calculated using the segmentation prediction map, edge-aware map, segmentation-annotated image, and edge-annotated image. An alternating parameter update strategy is used to alternately update the relevant weight coefficients during forward and backward propagation, thereby completing the overall optimization of the model and completing the training. Specifically:

[0061] Using the segmentation prediction map S, the edge-aware map E, and the segmentation annotation map S gt and edge annotation diagram E gt The segmentation loss and edge-aware loss are calculated, and then an alternating parameter update strategy is used to alternately update the relevant weight coefficients during forward and backward propagation to complete the overall optimization of the model, thereby completing the training. The loss calculation method is as follows:

[0062] Loss=λ1ζ CE (S,S gt )+λ2ζ BCE (E,E gt (17)

[0063] In the formula, ζ CE For cross-entropy loss, ζ BCE λ1 and λ2 are the weight coefficients of the segmentation loss and the edge perception loss, respectively.

[0064] Optionally, in step 3, the model segmentation effect is visualized and evaluation metrics are calculated using the test set, specifically as follows:

[0065] After the model training is completed, the test images in the test set are segmented pixel by pixel using the oil leakage segmentation model, and the intersection-union ratio (IU) between the model segmentation results and the segmentation annotation map is calculated.

[0066] According to specific embodiments provided by the present invention, the following technical effects are disclosed: The present invention provides a method for segmenting oil leakage in substation equipment based on dual-graph reasoning. This method includes constructing a dataset, using manually collected substation equipment inspection images as the data source, selecting inspection images of substation equipment containing oil leakage defects to construct the dataset, labeling the dataset using annotation software, dividing it into training and test sets, building an oil leakage segmentation model, iteratively training the model based on the training set, and visualizing the model's segmentation effect and calculating evaluation metrics using the test set. This method utilizes semantic graph reasoning to learn global texture relationships, mines structured texture features in fine-grained features, aggregates local region texture features, captures semantic adjacency relationships between aggregated features, models a semantic graph structure with aggregated features as nodes and semantic adjacency relationships as edges, and uses a graph convolutional network to complete node... This method infers semantic adjacency relationships to aggregate global contexts with similar texture information, enhancing intra-class consistency and inter-class differences, thereby improving missegmentation. It utilizes spatial graph reasoning to establish long-distance dependencies between edge pixels, capturing spatial adjacency relationships between pixels in fine-grained features. It models a spatial graph structure with pixels as nodes and spatial adjacency relationships as edges, and uses simplified graph convolution to infer spatial adjacency relationships between nodes, thus learning long-distance dependencies between edge pixels. An edge-aware auxiliary loss is set to apply regularization constraints to the model, thereby improving edge blurring caused by downsampling mechanisms and the diffusion characteristics of leaked oil. This method fully utilizes the texture and edge characteristics of leaked oil, solving the problems of difficult defect feature extraction, low segmentation accuracy, and poor segmentation effect caused by the strong correlation, varied morphology, and diffusion characteristics of leaked oil components. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a schematic diagram of the process for segmenting oil leakage in power equipment based on dual-graph reasoning, according to an embodiment of the present invention.

[0069] Figure 2 This is a schematic diagram of the semantic graph reasoning structure;

[0070] Figure 3 This is a schematic diagram of the spatial diagram reasoning structure;

[0071] Figure 4 This is a schematic diagram showing the image segmentation results of typical power equipment oil leakage before and after baseline model optimization. Detailed Implementation

[0072] The purpose of this invention is to provide a method for segmenting oil leakage in power equipment based on dual-graph reasoning. This method utilizes the texture and edge characteristics of the oil leakage to solve the problems of difficulty in extracting defect features, low segmentation accuracy, and poor segmentation effect caused by the strong correlation, variable morphology, and diffusion characteristics of oil leakage components.

[0073] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] like Figure 1 As shown in the figure, the method for segmenting oil leakage in power equipment based on dual-graph reasoning provided in this embodiment of the invention includes the following steps:

[0075] Step 1: Construct a dataset. Using manually collected images of substation equipment inspections as the data source, select images of substation equipment inspections containing oil leakage defects to construct a dataset. Use annotation software to annotate the dataset and divide it into training and test sets.

[0076] Step 2: Build an oil leakage segmentation model and complete iterative training of the model based on the training set;

[0077] Step 3: Use the test set to visualize the model segmentation effect and calculate the evaluation metrics.

[0078] In step 1, a dataset is constructed using manually collected images of substation equipment inspections as the data source. Images of substation equipment inspections containing oil leakage defects are selected to construct the dataset. The dataset is then labeled using annotation software, and divided into training and testing sets, as follows:

[0079] A dataset was constructed using manually collected images of substation equipment inspections as the data source. Images of substation equipment inspections containing oil leakage defects were selected. Referring to the construction method of the publicly available semantic segmentation dataset Cityscapes, the image resolution was unified to 2048×1024. Oil leakage areas in the substation equipment inspection images were manually annotated using annotation tools to obtain segmented and annotated images. Edge-annotated images were automatically generated using the Canny edge detection method. The original substation equipment inspection images, the corresponding segmented and annotated images, and the edge-annotated images were combined to form image samples. All image samples were integrated to construct the dataset. The training set and test set were divided in a 7:3 ratio. The substation equipment oil leakage segmentation dataset constructed in this embodiment of the invention contains 1360 sets of samples. Each set of samples consists of the original image, the segmented and annotated image, and the edge-annotated image. Among them, 952 sets of samples are used as the training set, and 408 sets of samples are used as the test set, with a training-to-test ratio of approximately 7:3.

[0080] Step 2 involves iterative training of the model based on the training set, specifically including the following steps:

[0081] Step 201: Extract multi-scale feature maps, including deep feature maps and shallow feature maps, through the feature extraction network of the oil leakage segmentation model. The deep feature maps are rich in semantic information, while the shallow feature maps are rich in spatial information.

[0082] Step 202: Concatenate and fuse the first two shallow feature maps. Use two-dimensional discrete wavelet transform to decompose the shallow feature map into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture. Concatenate and fuse the two sub-maps to obtain a structured texture feature map.

[0083] Step 203: Aggregate the texture information of the structured texture feature map by weighted average pooling. The texture information at the region level is represented by the feature vector of a single node, realizing the transformation from the structured texture feature map in the pixel space to the semantic node in the interaction space. By calculating the similarity coefficient between semantic nodes, a weighted adjacency matrix for information propagation between nodes is generated to complete the semantic graph representation. The node information aggregation and state update are realized by a two-layer graph convolutional neural network. After completing the semantic relationship representation and reasoning between nodes, the node matrix in the sparse space is mapped back to the pixel space to obtain the enhanced texture feature map.

[0084] Step 204: The high-resolution feature map aggregates spatial information at different scales through the Bottle2neck layer to obtain a node matrix. Guided by deep semantic features, the similarity coefficient between node feature vectors is calculated to obtain a weighted adjacency matrix, thus completing the spatial graph construction. The node information aggregation and state update are completed using the simplest graph convolution. After dimensional transformation, the enhanced edge feature map is obtained.

[0085] Step 205: Fuse the enhanced texture feature map with the shallow feature map extracted by the feature extraction network, fuse the enhanced edge feature map with the deep feature map extracted by the feature extraction network, and obtain the segmentation prediction map through the semantic segmentation head;

[0086] Step 206: Calculate the segmentation loss and edge perception loss using the segmentation prediction map, edge-aware map, segmentation labeled image, and edge-aware image. Use an alternating parameter update strategy to alternately update the relevant weight coefficients during forward and backward propagation to complete the overall optimization of the model and complete the training.

[0087] In step 201, multi-scale feature maps are extracted using the feature extraction network of the leaked oil segmentation model, including deep feature maps and shallow feature maps. The deep feature maps are rich in semantic information, while the shallow feature maps are rich in spatial information. Specifically:

[0088] Obtain the image to be segmented x, and use a dilated convolutional ResNet50 network to extract multi-scale feature maps {x0,x1,x2,x3,x4} from the image to be segmented x. Here, {x0,x1} are high-resolution shallow feature maps, rich in spatial information such as texture, color, and contour, while {x2,x3,x4} are low-resolution deep feature maps, rich in semantic information for scene understanding. It is worth noting that the multi-scale feature maps extracted by the original ResNet50 network do not include x0. In order to better mine the spatial information of shallow features, x0 is taken as the output feature map of the first three 3×3 convolutions of the ResNet50 network.

[0089] In step 202, the first two shallow feature maps are spliced ​​and fused. A two-dimensional discrete wavelet transform is used to decompose the shallow feature map into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture. The two sub-maps are then spliced ​​and fused to obtain a structured texture feature map, specifically:

[0090] The shallow feature map x1 is upsampled, concatenated with x0, and then compressed through a 1×1 convolution to obtain a high-resolution feature map. for:

[0091]

[0092] In the formula, f1(·,w1) is a 1×1 convolution operation with compressed channel dimensions, w1 is the learnable weight parameter in the 1×1 convolution, and (·) up This indicates an upsampling operation implemented by bilinear interpolation. This represents the feature map concatenation operation, which uses two-dimensional discrete wavelet transform to... spa Decomposed into low-frequency feature sub-maps reflecting smooth texture and high-frequency feature sub-maps reflecting abrupt texture changes and for:

[0093]

[0094] In the formula, These represent low-frequency feature sub-images, horizontal high-frequency feature sub-images, vertical high-frequency feature sub-images, and diagonal high-frequency feature sub-images, respectively. 2DWT represents the two-dimensional wavelet transform operation, and the selected wavelet basis function is Daubechies. The four sub-images are fused and concatenated, and the channel dimension is compressed through a 1×1 convolution to obtain the texture feature map. for:

[0095]

[0096] In the formula, f2(·,w2) is a 1×1 convolution operation that compresses the channel dimension, and w2 is the learnable weight parameter in the 1×1 convolution. This indicates a feature map splicing operation.

[0097] like Figure 2 As shown, in step 203, the texture information of the structured texture feature map is aggregated by weighted average pooling. The texture information at the region level is represented by the feature vector of a single node, realizing the transformation from the structured texture feature map in the pixel space to semantic nodes in the interaction space. By calculating the similarity coefficient between semantic nodes, a weighted adjacency matrix for information propagation between nodes is generated, completing the semantic graph representation. The node information aggregation and state update are realized through a two-layer graph convolutional neural network. After completing the semantic relationship representation and reasoning between nodes, the node matrix in the sparse space is mapped back to the pixel space to obtain the enhanced texture feature map, specifically:

[0098] Semantic graph reasoning is achieved through the semantic graph convolution module of the oil leakage segmentation model. First, a projection function Ω(·) is constructed to transform the structured texture feature map in the pixel space. Node matrix X mapped to the interaction space t ∈R N×N To achieve the aggregation of regional texture features, the projection function is represented as weighted average pooling:

[0099]

[0100] In the formula, This represents a set of learnable weight vectors. v represents the texture feature vector decomposed according to spatial dimensions. i ∈R N×1 The aggregated feature vector is represented by the projection function modeled as two convolution operations. With θ(·,w θ ), Used for dimensional transformation, θ(·,w) θ) is used to generate the weight vector b i The calculation of the node matrix is ​​simplified to:

[0101]

[0102] In the formula, This represents matrix multiplication. w θ Let be the learnable parameters in the two convolutional layers, (·) vec This indicates a matrix redirection operation, (·). T This represents the transpose operation. After mapping, modeling the semantic adjacency relationships between texture features in any region of the structured texture feature map is simplified to capturing the node matrix X. t The adjacency relationship between the feature vectors of the nodes is determined by two one-dimensional convolution operations φ1(·,w). φ1 ) and φ2(·,w φ2 The similarity coefficient between the feature vectors of captured nodes is used to measure the adjacency relationship between semantic nodes, resulting in a weighted adjacency matrix A. t ∈R N×N for:

[0103]

[0104] In the formula, w represents matrix multiplication. φ1 ,w φ2 Let be the learnable parameters in two one-dimensional convolutions, (·) T The transpose operation is represented. After constructing the node matrix and adjacency matrix, semantic graph node information aggregation and state update are implemented through a two-layer graph convolutional neural network (GCN). During training, the adjacency matrix learns the edge weights of node features, which reflect the global relationships of structured texture features in the dense pixel space. After information diffusion, each node is aggregated with all necessary node information, and its state is updated through a linear transformation, resulting in:

[0105]

[0106] In the formula, This represents the node matrix after the state update, and ReLU represents the ReLU activation operation. This represents the adjacency matrix smoothed using the degree matrix. To represent matrix multiplication, Laplace smoothing is replaced by the softmax activation operation. W represents the node matrix before the state update. (0) With W (1)Given the learnable weight matrices in the two GCN layers, after semantic graph inference is completed, a back-projection function is constructed. Following the approach of feature aggregation, the back projection function is also modeled as two convolution operations. With θ′(·,w θ′ ),in Used for dimension reduction, θ′(·,w θ′ Used for feature diffusion to enhance texture features. Characterized as:

[0107]

[0108] To simplify the calculation, θ′(x tex ,w θ′ Directly take θ(x) tex ,w θ The transpose of ) To restore the learnable parameters in the convolutional layer to the required dimensions, (·) maj This indicates a vector-to-matrix conversion operation.

[0109] like Figure 3 As shown, in step 204, the high-resolution feature map aggregates spatial information at different scales through the Bottle2neck layer to obtain a node matrix. Guided by deep semantic features, the similarity coefficient between node feature vectors is calculated to obtain a weighted adjacency matrix, completing the spatial graph construction. The node information aggregation and state update are then performed using the simplest graph convolution. After dimensionality transformation, an enhanced edge feature map is obtained, specifically:

[0110] Spatial graph inference is achieved through multiple cascaded spatial graph convolutional modules in the oil leakage segmentation model. Let t∈1,2,...,m be the positional encoding of the spatial graph convolution. Taking a single spatial graph inference module as an example, its input is a shallow feature map. With deep feature maps in Take the output feature map of the (t-1)th spatial map convolution to provide local contour detail features. When t=1, Take high-resolution feature map x spa , Sequentially extract the deep output feature maps x2, x3, and x4 from the backbone network, compress the channel dimension of both the deep and shallow feature maps to 1 using 1×1 convolutions, and then align the deep feature maps using upsampling operations. With shallow feature map The spatial dimensions are merged, spliced, and flattened to obtain a fused feature map. for:

[0111]

[0112] In the formula, f2(·,w2) and f3(·,w3) are 1×1 convolution operations that compress the channel dimension, w2 and w3 are the learnable weight parameters in the 1×1 convolution, and (·) up This indicates an upsampling operation implemented by bilinear interpolation. This indicates a feature map concatenation operation, (·) vec Indicates matrix steering operations to fuse feature maps. As input, the input feature map is processed using global adaptive average pooling. The spatial dimension is compressed to 1, and the weight coefficients between deep and shallow features are obtained through the sigmoid activation function. After diagonalization, a diagonal matrix Λ∈R is obtained. 2×2 It is important to note that the diagonal matrix is ​​not fixed; it is updated and adjusted during training with different sample inputs, ultimately forming the optimal linear combination of deep and shallow features. The diagonal matrix Λ∈R 2×2 for:

[0113]

[0114] In the formula, Sigmoid represents the sigmoid activation operation. express The feature vector obtained after global adaptive average pooling, where diag represents the diagonalization operation, is transformed using two 1×1 convolution-based transformation functions from the input feature map. Transform into and Perform a transpose operation on Q, calculate the similarity matrix using matrix multiplication, and then activate it using the softmax function to obtain the node feature vector x. i With node feature vector x j Normalized weights of spatial adjacency Calculate the normalized weighted adjacency matrix for:

[0115]

[0116] In the formula, Softmax represents the softmax activation operation. w represents matrix multiplication. φ3 ,w φ4 The parameters that can be learned in two one-dimensional convolutions, the node matrix Then from the shallow feature map After extracting multi-scale features through the Bottle2neck layer, the results are obtained by dimensionality transformation and transpose operation, i.e.:

[0117]

[0118] In the formula, μ(·,wμ ) indicates the use of the Bottle2neck layer, w μ This represents the learnable weight parameters in the Bottle2neck layer, (·). vec This represents a matrix-to-vector operation. After constructing the node matrix and adjacency matrix, it aggregates node information and updates the state of the spatial graph through a simplified graph convolution. A Laplacian smoothing operation is performed on the adjacency matrix to propagate node features across the spatial graph. After aggregating all necessary node information, the node state is updated using a learnable weight matrix, and enhanced edge features are obtained through dimensionality transformation. for:

[0119]

[0120] In the formula, ReLU represents the activation operation using ReLU, and D e It is the adjacency matrix A e The degree matrix W is used to perform the Laplace smoothing operation. e Learnable weight matrix Represents matrix multiplication, (·) mat This represents the vector-to-matrix operation, considering that the space complexity of the adjacency matrix calculation reaches O(HW). 2 This severely impacts end-to-end inference efficiency. To further simplify computation, the Laplacian smoothing operation is replaced by a softmax activation operation, with a learnable weight matrix W provided by one-dimensional convolution. e Furthermore, the large-scale adjacency matrix A is not separately calculated and stored during spatial graph reasoning. e Instead, it uses the association rules of matrix multiplication to make the adjacency matrix A e The space complexity is reduced to O(HW), and the calculation of enhanced edge features is simplified as follows:

[0121]

[0122] x spa Guided by deep semantic features x2, x3, and x4, three consecutive spatial graph convolutional modules are used to enable the system to perceive the details of the oil leakage edge from the perspective of spatial adjacency, ultimately resulting in enhanced edge features. After further compressing the channel dimension through 1×1 convolution and activating with the Sigmoid function, the edge-aware map E∈R is obtained. 1×H×W Simultaneously, 1×1 convolution is used to expand the channel dimension to obtain an enhanced edge feature map to supplement spatial information. for:

[0123]

[0124]

[0125] In the formula, f4(·,w4) is a 1×1 convolution operation with compressed channel dimensions, w4 is the learnable weight parameter in the 1×1 convolution, Sigmiod indicates activation operation using the sigmiod function, f5(·,w5) is a 1×1 convolution operation with compressed channel dimensions, and w5 is the learnable weight parameter in the 1×1 convolution.

[0126] In step 205, the enhanced texture feature map is fused with the shallow feature map extracted by the feature extraction network, and the enhanced edge feature map is fused with the deep feature map extracted by the feature extraction network. A segmentation prediction map is then obtained through the semantic segmentation head. Specifically:

[0127] Enhance texture feature map T graph The shallow feature map x1 extracted by the feature extraction network is fused together, and the edge feature map E is enhanced. s Compared with the deep feature map x after ASPP module aspp The feature maps are fused, and the fused shallow and deep feature maps are fed into the segmentation head to obtain the segmentation prediction map S∈R. 1×H×W .

[0128] In step 206, segmentation loss and edge-aware loss are calculated using the segmentation prediction map, edge-aware map, segmentation-annotated image, and edge-annotated image. An alternating parameter update strategy is used to alternately update the relevant weight coefficients during forward and backward propagation, thereby completing the overall optimization of the model and completing the training. Specifically:

[0129] Using the segmentation prediction map S, the edge-aware map E, and the segmentation annotation map S gt and edge annotation diagram E gt The segmentation loss and edge-aware loss are calculated, and then an alternating parameter update strategy is used to alternately update the relevant weight coefficients during forward and backward propagation to complete the overall optimization of the model, thereby completing the training. The loss calculation method is as follows:

[0130] Loss=λ1ζ CE (S,S gt )+λ2ζ BCE (E,E gt (17)

[0131] In the formula, ζ CE For cross-entropy loss, ζ BCE λ1 and λ2 are the weight coefficients of the segmentation loss and the edge perception loss, respectively.

[0132] In step 3, the model segmentation performance is visualized and evaluation metrics are calculated using the test set, specifically as follows:

[0133] After model training, the test images in the test set were segmented pixel-by-pixel using the oil leakage segmentation model, and the intersection-over-union (IOU) ratio between the model segmentation results and the segmentation annotation maps was calculated. The selected evaluation metric was the IOU of the oil leakage region. The baseline model used in the experiment was the DeeplabV3+ semantic segmentation network, with an IOU of 65.55%. After introducing the spatial graph inference branch, the IOU increased to 70.19%, and after introducing the semantic graph inference branch, the IOU increased to 71.62%. Simultaneously introducing both the spatial graph inference branch and the semantic graph inference branch further increased the IOU to 73.77%, representing an improvement of approximately 8.22% compared to the DeeplabV3+ network. This quantitatively demonstrates the effectiveness of the proposed method. A schematic diagram of the typical substation oil leakage image segmentation results before and after baseline model optimization is shown below. Figure 4 As shown, the black area represents the background region, the light gray area represents the oil leakage region, and the dark gray area represents the network with missegmentation regions. It can be seen that the method of the present invention has a better oil leakage segmentation effect and significantly improves the missegmentation problem of the baseline network, which qualitatively proves the effectiveness of the method.

[0134] This invention provides a method for segmenting oil leakage in substation equipment based on dual-graph reasoning. The method includes constructing a dataset using manually collected substation inspection images as the data source, selecting inspection images containing oil leakage defects to construct the dataset, labeling the dataset using annotation software, dividing it into training and test sets, building an oil leakage segmentation model, iteratively training the model based on the training set, and visualizing the model's segmentation effect and calculating evaluation metrics using the test set. This method utilizes semantic graph reasoning to learn global texture relationships, mining structured texture features from fine-grained features, aggregating local region texture features, capturing semantic adjacency relationships between aggregated features, modeling a semantic graph structure with aggregated features as nodes and semantic adjacency relationships as edges, and using a graph convolutional network to infer semantic adjacency relationships between nodes. By aggregating global contexts with similar texture information, this method enhances intra-class consistency and inter-class differences, thereby improving the missegmentation problem. It utilizes spatial graph inference to establish long-distance dependencies between edge pixels, capturing spatial adjacency relationships between pixels in fine-grained features. A spatial graph structure is modeled with pixels as nodes and spatial adjacency relationships as edges. Simplified graph convolution is used to infer spatial adjacency relationships between nodes, thus learning long-distance dependencies between edge pixels. An edge-aware auxiliary loss is set to apply regularization constraints to the model, thereby improving the edge blurring problem caused by downsampling mechanisms and the diffusion characteristics of leaked oil. This method fully utilizes the texture and edge characteristics of leaked oil, solving the problems of difficult defect feature extraction, low segmentation accuracy, and poor segmentation effect caused by the strong correlation, varied morphology, and diffusion characteristics of leaked oil components.

[0135] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A transformer equipment oil leakage segmentation method based on double graph reasoning, characterized by, Comprise the following steps: Step 1: constructing a data set, manually collecting substation equipment inspection pictures as a data source, selecting substation equipment inspection images containing oil leakage defects to construct a data set, using a labeling software to label the data set, and dividing the training set and the test set; Step 2: build a leakage oil segmentation model, complete the iterative training of the model based on the training set; Step 3: use the test set to complete the visualization of the model segmentation effect and the calculation of the evaluation index; In step 2, the model is trained based on the training set, which comprises the following steps: Step 201: extract multi-scale feature maps through the feature extraction network of the leakage oil segmentation model, including deep feature maps and shallow feature maps, wherein the deep feature maps are rich in semantic information, and the shallow feature maps are rich in spatial information; Step 202: splice and fuse the two shallow feature maps, use two-dimensional discrete wavelet transform to decompose the shallow feature map into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture, and splice and fuse the two sub-maps to obtain a structured texture feature map; Step 203: aggregate the texture information of the structured texture feature map by weighted average pooling, represent the regional level texture information by a single node feature vector, realize the conversion from the structured texture feature map in the pixel space to the semantic node in the interaction space, generate a weighted adjacency matrix for node information propagation by calculating the similarity coefficient between semantic nodes, complete the semantic graph representation, realize node information aggregation and state update through a two-layer graph convolutional neural network, and after completing the semantic relationship representation and reasoning between nodes, map the node matrix in the sparse space back to the pixel space to obtain an enhanced texture feature map; Step 204: the high-resolution feature map aggregates spatial information at different scales through the Bottle2neck layer to obtain a node matrix, calculates the similarity coefficient between node feature vectors under the guidance of deep semantic features to obtain a weighted adjacency matrix, completes the construction of the spatial graph, and uses the simplest graph convolution to complete node information aggregation and state update, and obtains an enhanced edge feature map after dimension transformation; Step 205: fuse the enhanced texture feature map with the shallow feature map extracted by the feature extraction network, fuse the enhanced edge feature map with the deep feature map extracted by the feature extraction network, and obtain a segmentation prediction map through a semantic segmentation head; Step 206: calculate the segmentation loss and edge perception loss through the segmentation prediction map, the edge perception map, the segmentation labeled image and the edge labeled image, use the alternating parameter update strategy to complete the related weight coefficient update in the forward and backward propagation alternately, complete the overall optimization of the model, and complete the training.

2. The transformer oil leakage segmentation method based on double-map reasoning according to claim 1, wherein, In step 1, the data set is constructed, the manually collected substation equipment inspection pictures are used as the data source, the substation equipment inspection images containing oil leakage defects are selected to construct the data set, the labeling software is used to label the data set, and the training set and the test set are divided, specifically: The dataset is constructed by manually collecting substation equipment inspection pictures as a data source, selecting substation equipment inspection images containing oil leakage defects, unifying the image resolution to 2048x1024, manually labeling the oil leakage area in the substation equipment inspection image through a labeling tool to obtain a segmentation labeled image, automatically generating an edge labeled image through a canny edge detection method, and combining the original substation equipment inspection image, the corresponding segmentation labeled image and the edge labeled image to form an image sample, and integrating all image samples to construct the dataset, and dividing the training set and the test set according to the ratio of 7:

3.

3. The transformer oil leakage segmentation method based on double-map reasoning according to claim 1, wherein, In step 201, multi-scale feature maps are extracted by the feature extraction network of the oil leakage segmentation model, including deep feature maps and shallow feature maps, wherein the deep feature maps are rich in semantic information, and the shallow feature maps are rich in spatial information, and specifically: The to-be-segmented image x is obtained, and the to-be-segmented image x is extracted by the ResNet50 network with dilated convolution to obtain multi-scale feature mappings {x0, x1, x2, x3, x4}, wherein {x0, x1} are high-resolution shallow feature maps rich in spatial information, and {x2, x3, x4} are low-resolution deep feature maps rich in semantic information of scene understanding, wherein x0 takes the output feature maps of the first three 3x3 convolutions of the ResNet50 network.

4. The transformer oil leakage segmentation method based on double-map reasoning according to claim 3, wherein, In step 202, the first two shallow feature maps are spliced and fused, and the shallow feature maps are decomposed into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture by two-dimensional discrete wavelet transform, and the two sub-maps are spliced and fused to obtain a structured texture feature map, and specifically: The shallow feature map x1 is up-sampled, and the high-resolution feature map is obtained by feature splicing with x0 and compressing the channel dimension through 1x1 convolution is: where f1(·, w1) is a 1×1 convolution operation in the compressed channel dimension, w1 is a learnable weight parameter in the 1×1 convolution, (·) up denotes an up-sampling operation implemented by bilinear interpolation, denotes a feature map concatenation operation, x spa is decomposed into a low-frequency feature sub-map reflecting smooth texture and a high-frequency feature sub-map reflecting abrupt texture, and is: In the formula, respectively represent low-frequency feature subgraph, horizontal high-frequency feature subgraph, vertical high-frequency feature subgraph and diagonal high-frequency feature subgraph, 2DWT represents a two-dimensional wavelet transform operation, a wavelet base function selected is Daubechies, four subgraphs are fused and spliced, and a channel dimension is compressed through 1*1 convolution to obtain a texture feature graph is: In the formula, f2(·, w2) is a 1×1 convolution operation in the compression channel dimension, w2 is a learnable weight parameter in the 1×1 convolution, represents a feature map splicing operation.

5. The transformer oil leakage segmentation method based on double-map reasoning according to claim 4, wherein, In step 203, the texture information of the structured texture feature map is aggregated by weighted average pooling, the regional-level texture information is represented by a single node feature vector, the conversion from the structured texture feature map in the pixel space to the semantic node in the interaction space is realized, the similarity coefficient between the semantic nodes is calculated to generate a weighted adjacency matrix for information propagation between nodes, the semantic graph representation is completed, and the node information aggregation and state updating are realized by a two-layer graph convolutional neural network. After completing the semantic relationship representation and reasoning between nodes, the node matrix in the sparse space is mapped back to the pixel space to obtain an enhanced texture feature map, and specifically: The semantic graph inference is realized by a semantic graph convolution module of a leaky oil segmentation model. First, a projection function Ω(·) is constructed to map the structured texture feature map in the pixel space to the node matrix X in the interaction space t ∈R N×N to realize the aggregation of regional texture features, and the projection function is characterized by weighted average pooling:​ wherein, denotes a set of learnable weight vectors, denotes a texture feature vector decomposed by spatial dimensions, v i ∈R N×1 denotes an aggregated feature vector, the projection function is modeled as two convolution operations and θ(·, w θ ), is used for dimension transformation, θ(·, w θ ) is used to generate weight vector b i simplifies the calculation of node matrix to: In the formula, denotes a matrix multiplication operation, w θ are the learnable parameters in two convolutional layers, (·) vec denotes a matrix-to-vector operation, (·) T denotes a transposition operation, after the mapping, the modeling of the semantic adjacency relationship between the texture features in any region of the structured texture feature map is simplified to capturing the adjacency relationship between the node feature vectors in the matrix X t , the adjacency relationship between the node feature vectors in the matrix X φ1 is measured by two one-dimensional convolutional operations φ1(·,w φ2 ) and φ2(·,w t ) to capture the similarity coefficients between the node feature vectors, and a weighted adjacency matrix A N×N is obtained: wherein, denotes matrix multiplication operation, w φ1 ,w φ2 are learnable parameters in two one-dimensional convolutions, (·) T denotes transpose operation, after the construction of node matrix and adjacency matrix, the semantic graph node information aggregation and state update are realized through two-layer graph convolutional neural network, in the training process, the adjacency matrix learns the edge weight of node feature, which reflects the global relationship of structured texture features in dense pixel space, after information diffusion, each node aggregates all necessary node information and updates its state through linear transformation, and the following is obtained: where, denotes the state updated node matrix, Relu denotes the relu activation operation, denotes the adjacency matrix with Laplacian smoothing by degree matrix, denotes the matrix multiplication operation, and Laplacian smoothing is replaced by softmax activation operation, denotes the state updated node matrix, W (0) and W (1) are the learnable weight matrices in the two GCN layers, and the back-projection function is constructed after the semantic graph inference According to the idea of feature aggregation, the back-projection function is also modeled as two convolution operations and θ'(·, w θ′ ), where is used for dimension reduction, θ'(·, w θ′ ) is used for feature diffusion to enhance the texture features is represented as: Simplify the calculation, θ'(x tex ,w θ′ ) directly take the transpose of θ(x tex ,w θ ), is the dimension reduction convolution layer, (·) maj denotes the vector to matrix operation.

6. The transformer oil leakage segmentation method based on double-map reasoning according to claim 5, wherein, In step 204, the high-resolution feature map aggregates spatial information at different scales through the Bottle2neck layer to obtain a node matrix, and calculates the similarity coefficient between the node feature vectors under the guidance of the deep semantic feature to obtain a weighted adjacency matrix, completes the construction of the spatial graph, and completes the node information aggregation and state updating by using the simplest graph convolution. After dimension transformation, an enhanced edge feature map is obtained, and specifically: The spatial graph reasoning is realized by a plurality of serial spatial graph convolution modules of the leakage oil segmentation model, take t e 1, 2,..., m as the position code of the spatial graph convolution, taking a single spatial graph reasoning module as an example, the input is a shallow feature map With a deep feature map Wherein Take the output feature map of the t-1th spatial graph convolution, to provide local contour detail features, when t = 1, Take the high-resolution feature map x spa , Take the deep output feature maps x2, x3, x4 of the backbone network in turn, compress the channel dimensions of the deep feature map and the shallow feature map to 1 by using 1x1 convolution, and align the spatial dimensions of the deep feature map With the shallow feature map Fusion splicing and flattening the spatial dimensions to obtain the fusion feature map : In the formula, f2(·,w2) and f3(·,w3) are 1×1 convolution operations that compress the channel dimension, w2 and w3 are the learnable weight parameters in the 1×1 convolution, and (·) up This indicates an upsampling operation implemented by bilinear interpolation. This indicates a feature map concatenation operation, (·) vec Indicates matrix steering operations to fuse feature maps. As input, the input feature map is processed using global adaptive average pooling. The spatial dimension is compressed to 1, and the weight coefficients between deep and shallow features are obtained through the sigmoid activation function. After diagonalization, a diagonal matrix Λ∈R is obtained. 2×2 ,for: In the formula, Sigmoid represents a sigmoid activation operation, represents The feature vector obtained after global adaptive average pooling, diag represents diagonalization operation, and the input feature map is transformed into and The transpose operation is performed on Q, the similarity matrix is calculated by matrix multiplication operation, and the node feature vector x is obtained after activation by the softmax function. i The normalized weight value of the spatial adjacency relationship of the node feature vector x j The normalized weighted adjacency matrix is:​ where Softmax denotes a softmax activation operation, denotes a matrix multiplication operation, w φ3 ,w φ4 are learnable parameters in two one-dimensional convolutions, the node matrix then the shallow feature map After extracting multi-scale features through the Bottle2neck layer, dimension transformation and transpose operation are used to transform, i.e. In the formula, μ (·, w μ ) represents the use of Bottle2neck layer, w μ represents the learnable weight parameters in Bottle2neck layer, (·) vec represents the matrix to vector operation, after the construction of node matrix and adjacency matrix, the space graph node information aggregation and state update are realized through the simplest graph convolution form, the Laplace smoothing operation is performed on the adjacency matrix, the node features are propagated on the space graph, after aggregating all necessary node information, the node state is updated through the learnable weight matrix, and the enhanced edge features are obtained through dimension transformation is: In the formula, Relu represents using the relu activation operation, D e is the degree matrix of the adjacency matrix A e for performing Laplacian smoothing operation, W e is a learnable weight matrix, represents the matrix multiplication operation, (·) mat represents the vector conversion matrix operation, and the calculation of the enhanced edge feature is simplified as: x spa Through the continuous three spatial graph convolution modules guided by the deep semantic features x2, x3, x4, it perceives the edge details of the leaked oil from the perspective of spatial adjacency relationship, and finally obtains the enhanced edge feature Through 1×1 convolution, the channel dimension is compressed again, and after the Sigmoid function is activated, the edge perception graph E∈R is obtained 1×H×W At the same time, the 1×1 convolution is used to expand the channel dimension to obtain the enhanced edge feature graph for supplementing spatial information For: In the formula, f4(·, w4) is a 1x1 convolution operation for compressing the channel dimension, w4 is a learnable weight parameter in the 1x1 convolution, Sigmiod represents an activation operation using a sigmiod function, and f5(·, w5) is a 1x1 convolution operation for compressing the channel dimension, w5 is a learnable weight parameter in the 1x1 convolution.

7. The transformer oil leakage segmentation method based on double-map reasoning according to claim 6, wherein, In step 205, the enhanced texture feature map is fused with the shallow feature map extracted by the feature extraction network, the enhanced edge feature map is fused with the deep feature map extracted by the feature extraction network, and the segmentation prediction map is obtained through the semantic segmentation head, specifically: The enhanced texture feature map T graph is fused with the shallow feature map x1 extracted by the feature extraction network, and the enhanced edge feature map E s is fused with the deep feature map x aspp that has passed through the ASPP module, and the fused shallow feature map and the deep feature map are input into the segmentation head to obtain a segmentation prediction map S ∈ R 1×H×W .

8. The transformer oil leakage segmentation method based on double-map reasoning according to claim 7, wherein, In step 206, the segmentation loss and the edge perception loss are calculated through the segmentation prediction map, the edge perception map, the segmentation labeled image and the edge labeled image, the related weight coefficient update is alternately completed in the forward and backward propagation by using the alternating parameter update strategy, the overall optimization of the model is completed, the training is completed, and specifically: By segmenting the prediction map S, the edge-aware map E and the segmentation label map S gt and the edge label map E gt The segmentation loss and the edge-aware loss are calculated, and then the related weight coefficient update is alternately completed in the forward and backward propagation by using an alternating parameter update strategy, the overall optimization of the model is completed, so that the training is completed, wherein the loss calculation method is: Loss = λ1ζ CE (S,S gt )+λ2ζ BCE (E,E gt ) (17) where ζ CE is the cross-entropy loss, ζ BCE is the binary cross-entropy loss, and λ1and λ2are the weight coefficients of the segmentation loss and the edge-aware loss, respectively.

9. The transformer oil leakage segmentation method based on double-map reasoning according to claim 8, wherein, In step 3, the visualization of the segmentation effect of the model and the calculation of the evaluation index are completed by using the test set, specifically: After the model training is completed, the test image of the test set is segmented pixel by pixel through the oil leakage segmentation model, and the intersection over union between the model segmentation result and the segmentation labeled image is calculated.

Images