Circuit breaker image edge detection method fusing spatial constraint fuzzy clustering and lightweight network optimization

Abstract

The invention discloses a circuit breaker image edge detection method fusing spatial constraint fuzzy clustering and lightweight network optimization, and the method carries out the local adaptive threshold calculation through combining spatial constraint FCM and Otsu algorithms, and optimizes the edge detection process of a Canny operator. According to the method, fuzzy classification is carried out on a circuit breaker image by adopting spatial constraint FCM to obtain a strong marginal probability graph; the circuit breaker image is subjected to block processing through local adaptive threshold calculation, a global threshold is generated through integration, and then the global threshold is input into a Canny operator for accurate edge extraction. In order to further improve the detection effect, a lightweight neural network PiDiNet is used to correct a Canny output image. According to the method, the edge detection precision of the circuit breaker image can be effectively improved, and the method is suitable for edge extraction tasks in high-noise and complex background environments.

Description

Technical Field

[0001] The present invention relates to a circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization, belonging to the field of detection technology. Background Art

[0002] Edge detection of circuit breaker images is one of the important technologies in fault detection and intelligent diagnosis of power equipment. As a key protection device in the power system, the reliability and working status monitoring of circuit breakers are crucial to ensuring the stability of the power system. However, in complex industrial environments, the quality of circuit breaker images is often affected by various factors, which makes traditional edge detection methods face huge challenges. Specifically, circuit breaker images may be affected by metal reflections, background dynamic blur, and other environmental interferences, resulting in unclear edge information in the image and even the inability to extract valid edge data. These problems make traditional edge detection algorithms, such as the Canny operator, unable to perform stable and accurate edge extraction in complex environments. Therefore, how to accurately extract edge information of circuit breakers in complex industrial scenarios has become a key challenge in current research.

[0003] Traditional edge detection algorithms, such as the Canny operator, are not ideal for complex environments. The Canny operator typically relies on two thresholds (high and low) to determine whether a pixel belongs to an edge. Based on the threshold settings, the Canny operator can identify edges in an image. However, for circuit breaker images, due to the potential for significant noise, reflections, and blur, relying solely on traditional threshold settings often results in inaccurate edge extraction or loss of important edge information. Summary of the Invention

[0004] The purpose of the present invention is to provide a circuit breaker image edge detection method that integrates spatially constrained fuzzy clustering and lightweight network optimization. It is an improved edge detection method that combines the advantages of the spatially constrained fuzzy clustering algorithm (FCM) and the lightweight network PiDiNet. It is suitable for circuit breaker edge detection in complex industrial scenarios and can effectively deal with interference factors such as metal reflections, dynamic blurred backgrounds and environmental noise. The present invention combines the threshold of the adaptive canny operator with the spatially constrained FCM and the local adaptive Otsu algorithm, so that the algorithm can adapt to local changes in the image more flexibly, thereby improving the edge extraction accuracy. The use of the improved canny operator in combination with PiDiNet can significantly reduce the difficulty of network correction and provide continuous initial edges, suppress the noise areas that PiDiNet is sensitive to, and make image extraction more robust.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization includes the following steps:

[0007] Step 1: Data preprocessing:

[0008] Images of circuit breakers under different lighting conditions are acquired and grayscaled. The three-channel RGB images are converted into single-channel grayscale images. Gaussian filtering is performed on the grayscale images to eliminate noise and improve image quality. The images are input into the Scharr operator to obtain the gradient amplitude distribution of the images.

[0009] Step 2: Improve the canny operator edge detection:

[0010] Spatially constrained FCM is used to perform fuzzy classification on the circuit breaker image to obtain a strong edge probability map. A local adaptive threshold calculation is performed by combining spatially constrained FCM (fuzzy C-means clustering) with the Otsu algorithm. The circuit breaker image is then partitioned and integrated to generate a global threshold. This global threshold is then input into the Canny operator for precise edge extraction.

[0011] Step 3 uses a lightweight network to correct the output of the improved canny operator:

[0012] The pixel difference network PiDiNet is used to perform edge correction by comparing the differences between pixels in the image, avoiding excessively deep layers, reducing network depth, and improving computational efficiency.

[0013] The purpose of the present invention can be further achieved by the following technical measures:

[0014] Furthermore, the method of inputting the image into the Scharr operator to obtain the gradient amplitude distribution of the image in step 1 is:

[0015] The convolution kernels of the Scharr operator in the x and y directions are:

[0016]

[0017] Convolve with the image respectively to get the horizontal direction G x and vertical direction G y The gradient component of :

[0018] I x =I*G x

[0019] I y =I*G y

[0020] Among them, I x and Iy Represent the gradient of the image in the horizontal and vertical directions respectively, and * represents the convolution operation;

[0021] Finally, the gradient magnitude of the image is calculated:

[0022]

[0023] And calculate the gradient direction:

[0024]

[0025] Among them, G reflects the edge strength of the pixel point, and the larger the value, the more obvious the edge; θ represents the edge direction.

[0026] Furthermore, the specific method of step 2 is:

[0027] Adaptive dynamic threshold adjustment is achieved by combining spatially constrained FCM with Otsu threshold.

[0028] The spatially constrained FCM is used to generate a fuzzy membership map based on the strong and weak edge probabilities. By introducing the membership information of the domain pixels, the spatially adjacent pixels are forced to have similar cluster memberships. The spatial constraint term is added to the objective function of the traditional FCM:

[0029]

[0030] Among them, the first term is the clustering goal of traditional FCM, which minimizes the distance between each data point and its cluster center; the second term is the spatial constraint term, which is used to make the cluster membership of adjacent pixels similar to reduce noise interference and improve the robustness of edge detection; N is the total number of pixels, C is the number of clusters (taken as 2, i.e. edge and non-edge), u ij is the membership degree of pixel i to cluster j, m is the fuzzy factor, N i is the neighborhood of pixel i, α is the spatial constraint weight;

[0031] Initially, a membership matrix U is randomly generated. The histogram of the gradient amplitude is calculated based on the input gradient amplitude map, and the 25% and 75% quantiles of the gradient amplitude are used as the initial cluster centers v1 and v2 of the background and edge regions. At the same time, an elliptical adaptive neighborhood is designed based on the gradient direction of each pixel. The long axis of the neighborhood is along the gradient direction, which means that the information in the edge direction is more important; the short axis is perpendicular to the gradient direction, which means that the change in the edge normal direction is smaller. The size of the neighborhood is adaptively adjusted according to the gradient amplitude of the image and is proportional to the gradient amplitude:

[0032] a=α·G(x,y)

[0033] b=β·G(x,y)

[0034] Where a and b are the major and minor axis sizes of the ellipse, G(x,y) is the gradient amplitude of each pixel, α and β are scaling factors. The larger the gradient amplitude, the longer the major axis a. To avoid the loss of details caused by an overly large neighborhood when the gradient amplitude is too large, a maximum scale threshold is set to limit the maximum scale of the neighborhood:

[0035] a=min(α·G(x,y),a max )

[0036] b=min(β·G(x,y),b max )

[0037] The neighborhood generated by each pixel is iteratively optimized to update the membership u ij :

[0038]

[0039] And update the cluster center v j :

[0040]

[0041] The numerator of the membership formula is the distance from the current pixel to the cluster center plus the membership difference of its neighborhood; the denominator represents the ratio of the distance to all cluster centers;

[0042] The processed output fuzzy membership map contains two values, representing the probability of each pixel belonging to a strong edge or a weak edge. A local adaptive threshold is used to divide the image into blocks. Otsu selects the optimal threshold by maximizing the inter-class variance of foreground and background, thereby ensuring the highest discrimination between foreground and background.

[0043] The strong edge membership graph U strong Treat it as a probability grayscale image and calculate the Otsu threshold independently for each block; extract the membership value within the sub-block and construct the integral image S(x, y), which represents the integral image value at each position of the image I(x, y) as the sum of all pixels from (1, 1) to (x, y):

[0044] S(x,y)=I(x,y)+S(x-1,y)+S(x,y-1)+S(x-1,y-1)

[0045] Here S(x,y) represents the sum of the pixels in the rectangular area from the upper left corner of the image to the current pixel. The integral graph can be used to quickly calculate the cumulative probability of each gray level; I(x,y) is the pixel value of image I at position (x,y);

[0046] Calculate the grayscale distribution within the block:

[0047]

[0048] Among them, p i is the probability of each gray level i, N is the total number of pixels in the image, n i is the frequency of occurrence of gray level i, and the cumulative frequency of pixels at this gray level n i The calculation is as follows:

[0049] n i =S(i)

[0050] Where S(i) represents the cumulative sum of pixels in the image whose grayscale value is less than or equal to i:

[0051]

[0052] in, is an indicator function, which is 1 if I(x,y)≤i, otherwise it is 0. H and W are the height and width of the input image respectively;

[0053] Traverse all candidate thresholds T and select the one that makes the between-class variance The maximum threshold T local :

[0054]

[0055] Among them, ω0(T) and ω1(T) are the cumulative probabilities of the background class and the foreground class, respectively, and the calculation formula is as follows:

[0056]

[0057] ω1(T)=1-ω0(T)

[0058] The mean values of the background and foreground classes are:

[0059]

[0060] Between-class variance for:

[0061]

[0062] According to the calculated local threshold T local , the pixels in the sub-block are divided into strong edges (foreground) and weak edges (background), the strong edge is U strong ≥T local , weak edge is U strong <T local ; Set the local threshold T of all sub-blocks local Sort and take the median as the high threshold T H :

[0063] T H=Median({T local})

[0064] For the strong edge membership graph U strong Perform histogram analysis, extract its distribution characteristics, and dynamically calculate TL based on the distribution characteristics:

[0065] T L =μ weak +k·σ weak

[0066] Among them, μ weak is the mean membership value of the weak edge region, σ weak is the standard deviation of membership in the weak edge region, k is the empirical coefficient, and the global high and low thresholds T of the image are obtained. H and T L ;

[0067] The obtained T H 、T L Input the canny operator for double threshold processing and edge connection. If the gradient amplitude G(x,y) is greater than T H , then the pixel is considered to be an edge pixel; if the gradient amplitude G(x,y) is less than T L , then the pixel is considered not an edge pixel; if the gradient amplitude G(x,y) is between the two, it is considered a weak edge, and the neighboring pixels in the gradient direction are checked to determine whether the pixel is an edge pixel;

[0068] From the previously calculated gradient:

[0069]

[0070] The directions are quantized to 0°, 45°, 90°, and 135°; depending on the gradient direction, only the neighborhoods related to that direction are checked instead of all 8 neighborhoods:

[0071] 0° direction: along the horizontal direction, check (x,y-1) and (x,y+1)

[0072] 45° direction: along the diagonal line, check (x-1,y+1) and (x+1,y-1)

[0073] 90° direction: along the vertical direction, check (x-1,y) and (x+1,y)

[0074] 135° direction: along the other diagonal line, check (x-1, y-1) and (x+1, y+1)

[0075] If the adjacent pixels in the gradient direction are strong edges (i.e. G(x,y) is greater than T H), the pixel is considered an edge pixel, otherwise it is considered not an edge pixel; this prevents isolated noise pixels from being considered as edges while ensuring the integrity of the true edge;

[0076] The continuity of the edge is further enhanced by dilation and thinning. Dilation fills small breaks in the edge, and a circular structuring element S is selected and thinned to remove excess noise edges:

[0077]

[0078] Among them, E is the binary edge map, Represents the morphological dilation operation.

[0079] Perform erosion on the expanded image:

[0080]

[0081] Represents the morphological corrosion operation, and the final result is a refined edge image.

[0082] Furthermore, the specific method of step 3 is:

[0083] The PiDiNet network replaces the traditional convolution operation with pixel difference convolution (PDC):

[0084]

[0085] Among them, x center is the center pixel value, represents the neighborhood of the pixel, k*k is the size of the convolution kernel, ω ij is the weight of the convolution kernel, x ij is the input pixel value;

[0086] The PiDiNet network is composed of multiple stacked PDC modules, including shallow PDC and deep PDC. The shallow PDC uses a 3*3 convolution kernel to extract fine-grained edges; the deep PDC convolution kernel has a larger receptive field and uses dilated convolution to see farther pixels; dilated convolution inserts holes (dilation rate) between convolution kernel pixels, sampling pixels in a jumpy manner, thereby increasing the receptive field:

[0087]

[0088] Where y(i) is the pixel value on the output feature map, x(i) is the pixel value on the input feature map, w(k) is the weight of the convolution kernel, K is the size of the convolution kernel, and r is the dilation rate, which represents the spacing between convolution kernel elements. By increasing r, the convolution kernel can be sampled "jumpily," thereby expanding the receptive field. This paper uses a 3*3 dilated convolution kernel with a dilation rate of r = 2 to capture coarse-grained contours. This captures coarse and fine-grained edge contours and gradually extracts multi-scale edge features. After fusion, the edge prediction value is output.

[0089] PDC captures multi-angle edge information by four sets of orthogonal differential directions, covering 0° (horizontal direction), 90° (vertical direction), 45°, and 135° (diagonal direction). The network automatically adjusts the edge response strength in different directions through training, and calculates the weighted sum of the local pixel differences of the feature map in the direction θ to obtain gradient information:

[0090]

[0091] Among them, F θ (x,y) is the output feature value of position (x,y) in the θ direction, c is the index of the input feature channel, is the neighborhood range of the convolution kernel (such as the coordinate set of a 3*3 convolution kernel), is the learnable convolution kernel weight in direction θ, acting on the cth channel, with an offset of (i, j), I (c) (x+i, y+j) is the value of the cth channel of the input feature map at position (x+i, y+j), I (c) (x,y) is the value of the c-th channel of the input feature map at the center position (x,y);

[0092] PDC extracts local gradient information through differential kernels in multiple directions. Each direction corresponds to a set of independent parameters. The horizontal direction detects vertical edges and only calculates the pixel differences in the horizontal neighborhood and takes the offset:

[0093] (i,j)={(0,1),(0,-1)}

[0094] Detect horizontal edges in the vertical direction, calculate the pixel difference of the vertical neighborhood, and take the offset:

[0095] (i,j)={(1,0),(-1,0)}

[0096] Detect diagonal edges and take the offset:

[0097] (i,j) 45° ={(1,1),(-1,-1)}

[0098] (i,j) 135° ={(1,-1),(-1,1)}

[0099] The difference results of the four groups of directions are spliced by channel to form a multi-directional gradient feature map:

[0100] F PDC =Concat(F 0° ,F 90° ,F 45° ,F 135° )

[0101] Among them, F 角度 is the differential feature map for each direction, F PDC For the fused multi-directional features, the concat function represents the concatenation of feature maps along the channel dimension;

[0102] In order to reduce the amount of calculation and parameters in the feature fusion process, the convolution structure of the original PiDiNet network is improved. PDC uses depth-wise separable convolution for the spliced features, which greatly reduces the number of parameters while ensuring the same calculation effect.

[0103] The depth-wise separable convolution is divided into two independent parts: depth-wise convolution and point-wise convolution. The depth-wise convolution processes the information of each channel independently without mixing channel information. For each channel c and position (x, y), the output is D (c) (x,y) is:

[0104]

[0105] Among them, the depth convolution kernel K is the convolution kernel size of k*k, each channel c corresponds to an independent kernel K, I is the input feature map, H is the height, W is the width, and C is the number of input channels;

[0106] Point-by-point convolution uses a 1*1 convolution kernel to fuse channel information and adjust the output dimension. The input is the output D of the depth convolution. For each output channel m and position (x, y), the final output is O (m) (x,y) is:

[0107]

[0108] in, is the point-by-point convolution kernel, M is the number of output channels, and the output feature map

[0109] To avoid irrelevant noise interference and enhance key edge features, the original PDC is improved by introducing an attention mechanism in the skip connection part to enhance the fusion quality of features at different levels. The attention mechanism is divided into channel attention (CA) and spatial attention (SA), which adjust the importance of features in the channel dimension and spatial dimension respectively. Channel attention learns the importance weight of each channel and generates channel weights through global average pooling and fully connected layers. The global average pooling formula is as follows:

[0110]

[0111] Where H and W are the height and width respectively, and F(i,j,c) represents the value of the c-th channel at position (i,j) in the input feature map. A fully connected layer (FC) is then used to learn the importance of each channel and generate the channel weights CA. Finally, the generated channel attention weights are used to adjust the channels of the feature map to obtain a weighted feature map. The important channel features are then added through channel multiplication.

[0112] Spatial attention assigns a weight to each position based on the importance of local information in the feature map. Features at different positions contain different local information of the image. By learning the importance weight of each position in the feature map, spatial attention can enhance the features of key areas. Spatial attention generates position weights through spatial convolution. First, the channel dimension of the input feature map F is compressed:

[0113]

[0114] Among them, F avg It is the result of channel dimension average pooling, and each pixel takes the average value of all channels. max is the result of maximum pooling. Each pixel takes the maximum value of all channels. C is the number of channels. F(:,:,c) represents the index of the matrix, which means taking out all pixel values of the cth channel. The returned result is a two-dimensional matrix of H×W. avg and F max After concatenation, a 7*7 convolutional layer is used to learn their spatial distribution. Finally, the generated spatial attention map is applied to the original feature map F to obtain the weighted spatial attention result.

[0115] The deep features are upsampled to the original size by bilinear interpolation. Bilinear interpolation estimates the pixel value of the new position by weighted average of four adjacent pixels. Let the original feature map be F deep , given a new position (x,y), its interpolation is calculated as:

[0116] F′(x,y)=(1-u)(1-v)F(i,j)+u(1-v)F(i+1,j)+(1-u)vF(i,j+1)+uvF(i+1,j+1)

[0117] Where (i, j) is the integer coordinate closest to the upper left corner of (x, y), u = x–i, represents the normalized offset of x between i and i+1, v = y–j, represents the normalized offset of y between j and j+1, F(i, j) is the upper left pixel value, F(i+1, j) is the upper right pixel value, F(i, j+1) is the lower left pixel value, and F(i+1, j+1) is the lower right pixel value. The four terms represent the weighted contributions of the upper left, upper right, lower left, and lower right pixels, respectively. During the interpolation process, the input and output corners are strictly aligned, and the coordinate mapping range is consistent with the input size.

[0118] By adding the weighted results of the attention mechanism to the deep features after bilinear interpolation upsampling to the original size, the fused features can be obtained:

[0119] F fuse =Upsample(F deep )+(F low ·CA(F low )·SA(F low ))

[0120] Among them, F fuse is the fusion feature, F deep is the deep feature, F low is a shallow feature, CA(F low ) represents the weighted channel attention mechanism, SA(F low ) represents the weighted spatial attention mechanism, Upsample(F deep ) is the deep feature upsampled to the same size as the shallow feature through bilinear interpolation;

[0121] In the final output layer, the fused features are mapped to an edge probability map through 1*1 convolution:

[0122] E pred =σ(Conv 1*1 (F fuse ))

[0123] Among them, σ is the Sigmoid function, the output range is [0,1], F fuse It is the sum of the up-sampled feature maps at different levels, and Conv represents 1*1 convolution;

[0124] Use weighted cross entropy loss (WCE) to supervise the training and enhance the supervision weight of edge pixels:

[0125]

[0126] in, is the edge pixel weight, and the non-edge pixel weight is 1-β; i is the true edge label of the i-th pixel (0 for non-edge or 1 for edge), p i is the predicted edge probability of the i-th pixel, and N is the total number of pixels in the batch.

[0127] Compared with the prior art, the present invention has the following beneficial effects:

[0128] The present invention provides an improved edge detection method, which combines the advantages of the spatially constrained fuzzy clustering algorithm (FCM) and the lightweight network PiDiNet. It is suitable for circuit breaker edge detection in complex industrial scenarios and can effectively deal with interference factors such as metal reflections, dynamic blurred backgrounds and environmental noise. The present invention combines the threshold of the adaptive canny operator with the spatially constrained FCM and the local adaptive Otsu algorithm, so that the algorithm can adapt to local changes in the image more flexibly, thereby improving the edge extraction accuracy. The use of the improved canny operator in combination with PiDiNet can significantly reduce the difficulty of network correction and provide continuous initial edges, suppress the noise areas that PiDiNet is sensitive to, make image extraction more robust, effectively improve the edge detection accuracy of circuit breaker images, and is suitable for edge extraction tasks in high noise and complex background environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 This is the overall flow chart of the algorithm;

[0130] Figure 2 This is the improved PiDiNet network flowchart. DETAILED DESCRIPTION

[0131] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0132] This paper is an improved canny algorithm with adaptive threshold, which uses spatial constraint FCM to extract the optimal threshold number required and uses local adaptive threshold for optimization. This paper adopts a hybrid framework, based on the output image of the improved canny operator, and uses the lightweight network PiDiNet to correct it, output a coherent edge image, and improve the robustness of the training image results. The specific algorithm is shown in the attached Figure 1 , mainly including the following steps:

[0133] Step 1: Data Preprocessing

[0134] The circuit breaker is photographed by a camera from different angles in the early morning, noon, and dusk to obtain images of the circuit breaker under different lighting conditions. The collected color image of the circuit breaker is grayscaled, and the three-channel RGB image is converted into a single-channel grayscale image to reduce computational complexity and retain edge information. The grayscale image is Gaussian filtered to eliminate noise and improve image quality. Subsequently, the image is input into the Scharr operator to obtain the gradient amplitude distribution of the image. The Scharr operator is a differential operator used to calculate image gradients and is more sensitive to subtle gradient changes during edge detection. The convolution kernels of the Scharr operator in the x and y directions are:

[0135]

[0136] Convolve with the image respectively to get the horizontal direction G x and vertical direction G y The gradient component of :

[0137] I x =I*G x

[0138] I y =I*G y

[0139] Among them, I x and I y Represent the gradients of the image in the horizontal and vertical directions respectively, and * represents the convolution operation.

[0140] Finally, the gradient magnitude of the image is calculated:

[0141]

[0142] And calculate the gradient direction:

[0143]

[0144] Among them, G reflects the edge strength of the pixel point, and the larger the value, the more obvious the edge; θ reflects the edge direction, such as θ = 0, which means the edge is horizontal.

[0145] Step 2: Improve canny operator edge detection

[0146] To solve the problem of high and low threshold selection of traditional canny operator, a method combining spatial constraint FCM and Otsu threshold is adopted to realize adaptive dynamic threshold adjustment.

[0147] The spatial constraint FCM is used to generate a fuzzy membership map based on the strong and weak edge probabilities. By introducing the membership information of the domain pixels, the spatially adjacent pixels are forced to have similar cluster membership. The spatial constraint term is added to the objective function of the traditional FCM:

[0148]

[0149] Among them, the first term is the clustering goal of traditional FCM, which is to minimize the distance between each data point and its cluster center; the second term is the spatial constraint term, which is used to make the cluster membership of adjacent pixels as similar as possible to reduce noise interference and improve the robustness of edge detection. N is the total number of pixels, C is the number of clusters (taken as 2, i.e. edge and non-edge), u ij is the membership degree of pixel i to cluster j, m is the fuzzy factor, N i is the neighborhood of pixel i, and α is the spatial constraint weight.

[0150] Initially, a membership matrix U is randomly generated. The gradient magnitude histogram is calculated based on the input gradient magnitude map, and the 25% and 75% quantiles of the gradient magnitude are used as the initial cluster centers v1 and v2 for the background and edge regions. At the same time, an elliptical adaptive neighborhood is designed using the gradient direction of each pixel. The long axis of the neighborhood is along the gradient direction, meaning that information in the edge direction is more important; the short axis is perpendicular to the gradient direction, meaning that changes in the edge normal direction are smaller. The size of the neighborhood is adaptively adjusted based on the gradient magnitude of the image and is proportional to the gradient magnitude:

[0151] a=α·G(x,y)

[0152] b=β·G(x,y)

[0153] Where a and b are the major and minor axes of the ellipse, G(x,y) is the gradient amplitude of each pixel, and α and β are scaling factors. The larger the gradient amplitude, the longer the major axis a. To avoid excessively large neighborhoods resulting in loss of detail when the gradient amplitude is too large, a maximum scale threshold is set to limit the maximum scale of the neighborhood:

[0154] a=min(α·G(x,y),a max )

[0155] b=min(β·G(x,y),b max )

[0156] The neighborhood generated by each pixel is iteratively optimized to update the membership u ij :

[0157]

[0158] And update the cluster center v j :

[0159]

[0160] The numerator of the membership formula is the distance from the current pixel to the cluster center plus the membership difference of its neighborhood; the denominator represents the ratio of the distance to all cluster centers.

[0161] The resulting fuzzy membership map contains two values, representing the probability of each pixel belonging to a strong edge or a weak edge. Using a local adaptive threshold to partition the image, Otsu selects the optimal threshold by maximizing the inter-class variance between foreground and background, ensuring maximum foreground-background differentiation.

[0162] The strong edge membership graph U strong Treat it as a probability grayscale image and calculate the Otsu threshold for each block independently. Extract the membership value within the sub-block and construct the integral image S(x, y). The integral image value at each position of the image I(x, y) is the sum of all pixels from (1, 1) to (x, y):

[0163] S(x,y)=I(x,y)+S(x-1,y)+S(x,y-1)+S(x-1,y-1)

[0164] Here, S(x,y) represents the sum of the pixels in the rectangular area from the upper left corner of the image to the current pixel. The integral graph can be used to quickly calculate the cumulative probability of each grayscale level. I(x,y) is the pixel value at position (x,y) in image I.

[0165] Calculate the grayscale distribution within the block:

[0166]

[0167] Among them, p i is the probability of each gray level i, N is the total number of pixels in the image, n i is the frequency of occurrence of gray level i, and the cumulative frequency of pixels at this gray level n i The calculation is as follows:

[0168] n i =S(i)

[0169] Where S(i) represents the cumulative sum of pixels in the image whose grayscale value is less than or equal to i:

[0170]

[0171] in, is an indicator function, which is 1 if I(x,y)≤i, otherwise it is 0. H and W are the height and width of the input image respectively.

[0172] Traverse all candidate thresholds T and select the one that makes the between-class variance The maximum threshold T local :

[0173]

[0174] Among them, ω0(T) and ω1(T) are the cumulative probabilities of the background class and the foreground class, respectively, and the calculation formula is as follows:

[0175]

[0176] ω1(T)=1-ω0(T)

[0177] The mean values of the background and foreground classes are:

[0178]

[0179] Between-class variance for:

[0180]

[0181] According to the calculated local threshold T local , the pixels in the sub-block are divided into strong edges (foreground) and weak edges (background), the strong edge is U strong ≥T local , weak edge is U strong <T local . Set the local threshold T of all sub-blocks local Sort and take the median as the high threshold T H :

[0182] T H =Median({T local})

[0183] For the strong edge membership graph U strong Perform histogram analysis, extract its distribution characteristics, and dynamically calculate TL based on the distribution characteristics:

[0184] T L =μ weak +k·σ weak

[0185] Among them, μ weak is the mean membership value of the weak edge region, σ weak is the standard deviation of membership in the weak edge region, and k is the empirical coefficient. Thus, the global high and low thresholds T of the image are obtained. H and T L .

[0186] The obtained T H 、T LInput the canny operator for double threshold processing and edge connection. If the gradient amplitude G(x,y) is greater than T H , then the pixel is considered to be an edge pixel; if the gradient amplitude G(x,y) is less than T L , then the pixel is considered not an edge pixel; if the gradient amplitude G(x,y) is between the two, it is considered a weak edge, and the neighboring pixels in the gradient direction are checked to determine whether the pixel is an edge pixel.

[0187] From the previously calculated gradient:

[0188]

[0189] The directions are quantized to 0°, 45°, 90°, and 135°. Depending on the gradient direction, only the neighborhoods related to that direction are checked instead of all 8 neighborhoods:

[0190] 0° direction: along the horizontal direction, check (x,y-1) and (x,y+1)

[0191] 45° direction: along the diagonal line, check (x-1,y+1) and (x+1,y-1)

[0192] 90° direction: along the vertical direction, check (x-1,y) and (x+1,y)

[0193] 135° direction: along the other diagonal line, check (x-1, y-1) and (x+1, y+1)

[0194] If the adjacent pixels in the gradient direction are strong edges (i.e. G(x,y) is greater than T H ), the pixel is considered an edge pixel, otherwise it is not considered an edge pixel. This can effectively prevent isolated noise pixels from being considered as edges while ensuring the integrity of the true edge.

[0195] The continuity of the edge is further enhanced by dilation and thinning. Dilation fills small breaks in the edge, and a circular structuring element S is selected and thinned to remove excess noise edges:

[0196]

[0197] Among them, E is the binary edge map, Represents the morphological dilation operation.

[0198] Perform erosion on the expanded image:

[0199]

[0200] Represents the morphological corrosion operation, and the final result is a refined edge image.

[0201] Step 3 Lightweight pixel difference network PiDiNet correction

[0202] A lightweight network is used to correct the output of the improved canny operator. The pixel difference network PiDiNet is used to perform edge correction by comparing the differences between pixels in the image, avoiding excessive layers, reducing network depth, and improving computational efficiency. The PiDiNet network replaces the traditional convolution operation with pixel difference convolution (PDC):

[0203]

[0204] Among them, x center is the center pixel value, represents the neighborhood of the pixel, k*k is the size of the convolution kernel, ω ij is the weight of the convolution kernel, x ij is the input pixel value.

[0205] The PiDiNet network is composed of multiple stacked PDC modules, including shallow PDC and deep PDC. The shallow PDC uses a 3*3 convolution kernel to extract fine-grained edges; the deep PDC convolution kernel has a larger receptive field and uses dilated convolution to see more distant pixels. Dilated convolution inserts holes (dilation rate) between convolution kernel pixels, sampling pixels in a jumpy manner, thereby increasing the receptive field:

[0206]

[0207] Here, y(i) is the pixel value on the output feature map, x(i) is the pixel value on the input feature map, w(k) is the weight of the convolution kernel, K is the kernel size, and r is the dilation rate, representing the spacing between kernel elements. Increasing r allows the kernel to "jump" in sampling, thereby expanding the receptive field. This paper uses a dilated convolution with a 3x3 kernel and a dilation rate of r = 2 to capture coarse-grained contours. This captures both coarse and fine-grained edge contours and gradually extracts multi-scale edge features. After fusion, the output is the edge prediction.

[0208] PDC captures multi-angle edge information by four sets of orthogonal differential directions, covering 0° (horizontal direction), 90° (vertical direction), 45°, and 135° (diagonal direction). The network automatically adjusts the edge response strength in different directions through training, and calculates the weighted sum of the local pixel differences of the feature map in the direction θ to obtain gradient information:

[0209]

[0210] Among them, F θ(x,y) is the output feature value of position (x,y) in the θ direction, c is the index of the input feature channel, is the neighborhood range of the convolution kernel (such as the coordinate set of a 3*3 convolution kernel), is the learnable convolution kernel weight in direction θ, acting on the cth channel, with an offset of (i, j), I (c) (x+i, y+j) is the value of the cth channel of the input feature map at position (x+i, y+j), I (c) (x,y) is the value of the cth channel of the input feature map at the center position (x,y).

[0211] PDC extracts local gradient information through differential kernels in multiple directions. Each direction corresponds to a set of independent parameters. The horizontal direction detects vertical edges and only calculates the pixel differences in the horizontal neighborhood and takes the offset:

[0212] (i,j)={(0,1),(0,-1)}

[0213] Detect horizontal edges in the vertical direction, calculate the pixel difference of the vertical neighborhood, and take the offset:

[0214] (i,j)={(1,0),(-1,0)}

[0215] Detect diagonal edges and take the offset:

[0216] (i,j) 45° ={(1,1),(-1,-1)}

[0217] (i,j) 135° ={(1,-1),(-1,1)}

[0218] The difference results of the four groups of directions are spliced by channel to form a multi-directional gradient feature map:

[0219] F PDC =Concat(F 0° ,F 90° ,F 45° ,F 135° )

[0220] Among them, F 角度 is the differential feature map for each direction, F PDC For the fused multi-directional features, the concat function represents the splicing feature map along the channel dimension.

[0221] In order to reduce the amount of calculation and parameters in the feature fusion process and improve the convolution structure of the original PiDiNet network, PDC uses depth-wise separable convolution for the spliced features, which greatly reduces the number of parameters while ensuring the same calculation effect.

[0222] Depthwise separable convolution is divided into two independent parts: depthwise convolution and pointwise convolution. Depthwise convolution processes the information of each channel independently without mixing channel information. For each channel c and position (x, y), it outputs D (c) (x,y) is:

[0223]

[0224] Among them, the depth convolution kernel K is the convolution kernel size of k*k, each channel c corresponds to an independent kernel K, I is the input feature map, H is the height, W is the width, and C is the number of input channels.

[0225] Point-by-point convolution uses a 1*1 convolution kernel to fuse channel information and adjust the output dimension. The input is the output D of the depth convolution. For each output channel m and position (x, y), the final output is O (m) (x,y) is:

[0226]

[0227] in, is the point-by-point convolution kernel, M is the number of output channels, and the output feature map

[0228] To avoid irrelevant noise interference and enhance key edge features, the original PDC is improved by introducing an attention mechanism in the skip connection part to enhance the fusion quality of features at different levels. The attention mechanism is divided into channel attention (CA) and spatial attention (SA), which adjust the importance of features in the channel dimension and spatial dimension respectively. Channel attention learns the importance weight of each channel and generates channel weights through global average pooling and fully connected layers. The global average pooling formula is as follows:

[0229]

[0230] Where H and W are the height and width, respectively, and F(i, j, c) represents the value of the c-th channel at position (i, j) in the input feature map. A fully connected layer (FC) is then used to learn the importance of each channel and generate the channel weights CA. Finally, the generated channel attention weights are used to adjust the channels of the feature map to obtain a weighted feature map. Important channel features are then added through channel multiplication.

[0231] Spatial attention assigns a weight to each position in the feature map based on the importance of the local information at that position. Features at different positions contain different local information of the image. By learning the importance weights for each position in the feature map, spatial attention can enhance the features of key areas. Spatial attention generates position weights through spatial convolution. First, the input feature map F is compressed into channel dimensions:

[0232]

[0233] Among them, F avg It is the result of channel dimension average pooling, and each pixel takes the average value of all channels. max is the result of maximum pooling. Each pixel takes the maximum value of all channels. C is the number of channels. F(:,:,c) represents the index of the matrix, which means taking out all pixel values of the cth channel. The returned result is a two-dimensional matrix of H×W. avg and F max After concatenation, a 7x7 convolutional layer is used to learn their spatial distribution. Finally, the generated spatial attention map is applied to the original feature map F to obtain the weighted spatial attention result.

[0234] The deep features are upsampled to the original size by bilinear interpolation. Bilinear interpolation estimates the pixel value of the new position by weighted average of four adjacent pixels. Let the original feature map be F deep , given a new position (x,y), its interpolation is calculated as:

[0235] F′(x,y)=(1-u)(1-v)F(i,j)+u(1-v)F(i+1,j)+(1-u)vF(i,j+1)+uvF(i+1,j+1)

[0236] Where (i, j) is the integer coordinate closest to the upper left corner of (x, y), u = x – i represents the normalized offset of x between i and i + 1, v = y – j represents the normalized offset of y between j and j + 1, F(i, j) is the value of the upper left pixel, F(i + 1, j) is the value of the upper right pixel, F(i, j + 1) is the value of the lower left pixel, and F(i + 1, j + 1) is the value of the lower right pixel. The four terms represent the weighted contributions of the upper left, upper right, lower left, and lower right pixels, respectively. During interpolation, the input and output corners are strictly aligned, and the coordinate mapping range is consistent with the input size.

[0237] By adding the weighted results of the attention mechanism to the deep features after bilinear interpolation upsampling to the original size, the fused features can be obtained:

[0238] F fuse =Upsample(F deep )+(F low·CA(F low )·SA(F low ))

[0239] Among them, F fuse is the fusion feature, F deep is the deep feature, F low is a shallow feature, CA(F low ) represents the weighted channel attention mechanism, SA(F low ) represents the weighted spatial attention mechanism, Upsample(F deep ) is the deep feature that is upsampled to the same size as the shallow feature through bilinear interpolation.

[0240] In the final output layer, the fused features are mapped to an edge probability map through 1*1 convolution:

[0241] E pred =σ(Conv 1*1 (F fuse ))

[0242] Among them, σ is the Sigmoid function, the output range is [0,1], F fuse It is the sum of the up-sampled feature maps at different levels, and Conv represents 1*1 convolution.

[0243] Use weighted cross entropy loss (WCE) to supervise the training and enhance the supervision weight of edge pixels:

[0244]

[0245] in, is the edge pixel weight, and the non-edge pixel weight is 1-β; i is the true edge label of the i-th pixel (0 for non-edge or 1 for edge), p i is the predicted edge probability of the i-th pixel, and N is the total number of pixels in the batch.

[0246] The final result is a continuous edge image obtained through joint training of the optimized Canny operator and the lightweight PiDiNet network. This method, combined with Canny's prior information, allows for faster convergence and greater robustness in PiDiNet training. This allows for more stable edge extraction in circuit breaker scenarios, reduces false detections caused by metal reflections, and improves edge continuity.

[0247] In addition to the above embodiments, the present invention may also have other implementation methods. Any technical solutions formed by equivalent replacement or equivalent transformation fall within the protection scope required by the present invention.

Claims

1. A circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization, characterized in that: The following steps are involved: Step 1: Data preprocessing: Images of circuit breakers under different lighting conditions are acquired and grayscaled. The three-channel RGB images are converted into single-channel grayscale images. Gaussian filtering is performed on the grayscale images to eliminate noise and improve image quality. The images are input into the Scharr operator to obtain the gradient amplitude distribution of the images. Step 2: Improve the canny operator edge detection: The spatially constrained FCM is used to perform fuzzy classification on the circuit breaker image and obtain a strong edge probability map. The spatially constrained FCM is combined with the Otsu algorithm to perform local adaptive threshold calculation. The circuit breaker image is then segmented and processed using the local adaptive threshold calculation. The global threshold is then integrated to generate a global threshold, which is then input into the Canny operator for precise edge extraction. Step 3 uses a lightweight network to correct the output of the improved canny operator: The pixel difference network PiDiNet is used to perform edge correction by comparing the differences between pixels in the image, avoiding excessively deep layers, reducing network depth, and improving computational efficiency.

2. The circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization according to claim 1 is characterized in that: The method of inputting the image into the Scharr operator to obtain the gradient amplitude distribution of the image in step 1 is: The convolution kernels of the Scharr operator in the x and y directions are: Convolve with the image respectively to get the horizontal direction G x and vertical direction G y The gradient component of : I x =I*G x I y =I*G y Among them, I x and I y Represent the gradients of the image in the horizontal and vertical directions respectively, and * represents the convolution operation; Finally, the gradient magnitude of the image is calculated: And calculate the gradient direction: Among them, G reflects the edge strength of the pixel point, and the larger the value, the more obvious the edge; θ represents the edge direction.

3. The circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization according to claim 1 is characterized in that: The specific method described in step 2 is: Adaptive dynamic threshold adjustment is achieved by combining spatially constrained FCM with Otsu threshold. The spatially constrained FCM is used to generate a fuzzy membership map based on the strong and weak edge probabilities. By introducing the membership information of the domain pixels, the spatially adjacent pixels are forced to have similar cluster memberships. The spatial constraint term is added to the objective function of the traditional FCM: Among them, the first term is the clustering goal of traditional FCM, which minimizes the distance between each data point and its cluster center; the second term is the spatial constraint term, which aims to make the cluster membership of adjacent pixels similar to reduce noise interference and improve the robustness of edge detection; N is the total number of pixels, C is the number of clusters, and u ij is the membership degree of pixel i to cluster j, m is the fuzzy factor, N i is the neighborhood of pixel i, α is the spatial constraint weight; Initially, a membership matrix U is randomly generated. The histogram of the gradient amplitude is calculated based on the input gradient amplitude map, and the 25% and 75% quantiles of the gradient amplitude are used as the initial cluster centers v1 and v2 of the background and edge regions. At the same time, an elliptical adaptive neighborhood is designed based on the gradient direction of each pixel. The long axis of the neighborhood is along the gradient direction, which means that the information in the edge direction is more important; the short axis is perpendicular to the gradient direction, which means that the change in the edge normal direction is smaller. The size of the neighborhood is adaptively adjusted according to the gradient amplitude of the image and is proportional to the gradient amplitude: a=α·G(x,y) b=β·G(x,y) Where a and b are the major and minor axis sizes of the ellipse, G(x,y) is the gradient amplitude of each pixel, α and β are scaling factors. The larger the gradient amplitude, the longer the major axis a. To avoid the loss of details caused by an overly large neighborhood when the gradient amplitude is too large, a maximum scale threshold is set to limit the maximum scale of the neighborhood: a=min(α·G(x,y),a max ) b=min(β·G(x,y),b max ) The neighborhood generated by each pixel is iteratively optimized to update the membership u ij : And update the cluster center v j : The numerator of the membership formula is the distance from the current pixel to the cluster center plus the membership difference of its neighborhood; the denominator represents the ratio of the distance to all cluster centers; The processed output fuzzy membership map contains two values, representing the probability of each pixel belonging to a strong edge or a weak edge. A local adaptive threshold is used to divide the image into blocks. Otsu selects the optimal threshold by maximizing the inter-class variance of foreground and background, thereby ensuring the highest discrimination between foreground and background. The strong edge membership graph U strong Treat it as a probability grayscale image and calculate the Otsu threshold independently for each block; extract the membership value within the sub-block and construct the integral image S(x, y), which represents the integral image value at each position of the image I(x, y) as the sum of all pixels from (1, 1) to (x, y): S(x,y)=I(x,y)+S(x-1,y)+S(x,y-1)+S(x-1,y-1) Here S(x,y) represents the sum of the pixels in the rectangular area from the upper left corner of the image to the current pixel. The integral graph can be used to quickly calculate the cumulative probability of each gray level; I(x,y) is the pixel value of image I at position (x,y); Calculate the grayscale distribution within the block: Among them, p i is the probability of each gray level i, N is the total number of pixels in the image, n i is the frequency of occurrence of gray level i, and the cumulative frequency of pixels at this gray level n i The calculation is as follows: n i =S(i) Where S(i) represents the cumulative sum of pixels in the image whose grayscale value is less than or equal to i: in, is an indicator function, which is 1 if I(x,y)≤i, otherwise it is 0; H and W are the height and width of the input image respectively; Traverse all candidate thresholds T and select the one that makes the between-class variance The maximum threshold T local : Among them, ω0(T) and ω1(T) are the cumulative probabilities of the background class and the foreground class, respectively, and the calculation formula is as follows: ω1(T)=1-ω0(T) The mean values of the background and foreground classes are: Between-class variance for: According to the calculated local threshold T local , the pixels in the sub-block are divided into strong edges and weak edges, the strong edge is U strong ≥T local , weak edge is U strong <T local ; Set the local threshold T of all sub-blocks local Sort and take the median as the high threshold T H : T H =Median({T local }) For the strong edge membership graph U strong Perform histogram analysis, extract its distribution characteristics, and dynamically calculate TL based on the distribution characteristics: T L =μ weak +k·s weak Among them, μ weak is the mean membership value of the weak edge region, σ weak is the standard deviation of membership in the weak edge region, k is the empirical coefficient, and the global high and low thresholds T of the image are obtained. H and T L ; The obtained T H 、T L Input the canny operator for double threshold processing and edge connection. If the gradient amplitude G(x,y) is greater than T H , then the pixel is considered to be an edge pixel; if the gradient amplitude G(x,y) is less than T L , then the pixel is considered not an edge pixel; if the gradient amplitude G(x,y) is between the two, it is considered a weak edge, and the neighboring pixels in the gradient direction are checked to determine whether the pixel is an edge pixel; From the previously calculated gradient: The directions are quantized to 0°, 45°, 90°, and 135°; depending on the gradient direction, only the neighborhoods related to that direction are checked instead of all 8 neighborhoods: 0° direction: along the horizontal direction, check (x,y-1) and (x,y+1) 45° direction: along the diagonal line, check (x-1,y+1) and (x+1,y-1) 90° direction: along the vertical direction, check (x-1,y) and (x+1,y) 135° direction: along the other diagonal line, check (x-1, y-1) and (x+1, y+1) If the adjacent pixel in the gradient direction is a strong edge, the pixel is considered an edge pixel, otherwise it is not an edge pixel; prevent isolated noise pixels from being regarded as edges while ensuring the integrity of the true edge; The continuity of the edge is further enhanced by dilation and thinning. Dilation fills small breaks in the edge, and a circular structuring element S is selected and thinned to remove excess noise edges: Among them, E is the binary edge map, represents the morphological dilation operation; Perform erosion on the expanded image: Represents the morphological corrosion operation, and the final result is a refined edge image.

4. The circuit breaker image edge detection method integrating spatially constrained fuzzy clustering and lightweight network optimization according to claim 1 is characterized in that: The specific method of step 3 is: The PiDiNet network replaces the traditional convolution operation with pixel differential convolution: Among them, x center is the center pixel value, represents the neighborhood of the pixel, k*k is the size of the convolution kernel, ω ij is the weight of the convolution kernel, x ij is the input pixel value; The PiDiNet network is composed of multiple stacked PDC modules, including shallow PDC and deep PDC. The shallow PDC uses a 3*3 convolution kernel to extract fine-grained edges; the deep PDC convolution kernel has a larger receptive field and uses dilated convolution to see farther pixels; dilated convolution inserts holes between convolution kernel pixels and samples pixels in a jumpy manner, thereby increasing the receptive field: Where y(i) is the pixel value on the output feature map, x(i) is the pixel value on the input feature map, w(k) is the weight of the convolution kernel, K is the size of the convolution kernel, and r is the dilation rate, which represents the spacing between convolution kernel elements. By increasing r, the convolution kernel can be sampled "jumpily", thereby expanding the receptive field. The edge contours of coarse and fine granularity are captured and multi-scale edge features are gradually extracted and fused, and the edge prediction value is output. PDC captures multi-angle edge information by four sets of orthogonal differential directions, covering 0°, 90°, 45°, and 135°. The network automatically adjusts the edge response strength in different directions through training, and calculates the weighted sum of the local pixel differences of the feature map in the direction θ to obtain the gradient information: Among them, F θ (x,y) is the output feature value of position (x,y) in the θ direction, c is the index of the input feature channel, is the neighborhood range of the convolution kernel, is the learnable convolution kernel weight in direction θ, acting on the cth channel, with an offset of (i, j), I (c) (x+i, y+j) is the value of the cth channel of the input feature map at position (x+i, y+j), I (c) (x,y) is the value of the c-th channel of the input feature map at the center position (x,y); PDC extracts local gradient information through differential kernels in multiple directions. Each direction corresponds to a set of independent parameters. The horizontal direction detects vertical edges and only calculates the pixel differences in the horizontal neighborhood and takes the offset: (i,j)={(0,1),(0,-1)} Detect horizontal edges in the vertical direction, calculate the pixel difference of the vertical neighborhood, and take the offset: (i,j)={(1,0),(-1,0)} Detect diagonal edges and take the offset: (i,j) 45° ={(1,1),(-1,-1)} (i,j) 135° ={(1,-1),(-1,1)} The difference results of the four groups of directions are spliced by channel to form a multi-directional gradient feature map: F PDC =Concat(F 0° ,F 90° ,F 45° ,F 135° ) Among them, F 角度 is the differential feature map for each direction, F PDC For the fused multi-directional features, the concat function represents the concatenation of feature maps along the channel dimension; In order to reduce the amount of calculation and parameters in the feature fusion process and improve the convolution structure of the original PiDiNet network, PDC uses depth-wise separable convolution for the spliced features; The depth-wise separable convolution is divided into two independent parts: depth-wise convolution and point-wise convolution. The depth-wise convolution processes the information of each channel independently without mixing channel information. For each channel c and position (x, y), the output is D (c) (x,y) is: Among them, the depth convolution kernel K is the convolution kernel size of k*k, each channel c corresponds to an independent kernel K, I is the input feature map, H is the height, W is the width, and C is the number of input channels; Point-by-point convolution uses a 1*1 convolution kernel to fuse channel information and adjust the output dimension. The input is the output D of the depth convolution. For each output channel m and position (x, y), the final output is O (m) (x,y) is: in, is the point-by-point convolution kernel, M is the number of output channels, and the output feature map In order to avoid irrelevant noise interference and enhance key edge features, the original PDC is improved by introducing an attention mechanism in the skip connection part to enhance the fusion quality of features at different levels. The attention mechanism is divided into channel attention and spatial attention, which adjust the importance of features in the channel dimension and spatial dimension respectively. Channel attention learns the importance weight of each channel and generates channel weights through global average pooling and fully connected layers. The global average pooling formula is as follows: Where H and W are the height and width respectively, and F(i, j, c) represents the value of the c-th channel at position (i, j) in the input feature map. A fully connected layer is then used to learn the importance of each channel and generate the channel weight CA. Finally, the generated channel attention weight is used to adjust the channels of the feature map to obtain a weighted feature map, and important channel features are added by channel multiplication. Spatial attention assigns a weight to each position based on the importance of local information in the feature map. Features at different positions contain different local information of the image. By learning the importance weight of each position in the feature map, spatial attention can enhance the features of key areas. Spatial attention generates position weights through spatial convolution. First, the channel dimension of the input feature map F is compressed: Among them, F avg It is the result of channel dimension average pooling, and each pixel takes the average value of all channels. max is the result of maximum pooling. Each pixel takes the maximum value of all channels. C is the number of channels. F(:,:,c) represents the index of the matrix, which means taking out all pixel values of the cth channel. The returned result is a two-dimensional matrix of H×W. avg and F max After splicing, a 7*7 convolutional layer is used to learn their spatial distribution. Finally, the generated spatial attention map is applied to the original feature map F to obtain the weighted spatial attention result. The deep features are upsampled to the original size by bilinear interpolation. Bilinear interpolation estimates the pixel value of the new position by weighted average of four adjacent pixels. Let the original feature map be F deep , given a new position (x,y), its interpolation is calculated as: F′(x,y)=(1-u)(1-v)F(i,j)+u(1-v)F(i+1,j)+(1-u)vF(i,j+1)+uvF(i+1,j+1) Where (i, j) is the integer coordinate closest to the upper left corner of (x, y), u = x–i, represents the normalized offset of x between i and i+1, v = y–j, represents the normalized offset of y between j and j+1, F(i, j) is the upper left pixel value, F(i+1, j) is the upper right pixel value, F(i, j+1) is the lower left pixel value, and F(i+1, j+1) is the lower right pixel value. The four terms represent the weighted contributions of the upper left, upper right, lower left, and lower right pixels, respectively. During the interpolation process, the input and output corners are strictly aligned, and the coordinate mapping range is consistent with the input size. By adding the weighted results of the attention mechanism to the deep features after bilinear interpolation upsampling to the original size, the fused features can be obtained: F fuse =Upsample(F deep )+(F low ·CA(F low )·SA(F low )) Among them, F fuse is the fusion feature, F deep is the deep feature, F low is a shallow feature, CA(F low ) represents the weighted channel attention mechanism, SA(F low ) represents the weighted spatial attention mechanism, Upsample(F deep ) is the deep feature upsampled to the same size as the shallow feature through bilinear interpolation; In the final output layer, the fused features are mapped to an edge probability map through 1*1 convolution: AND pred =σ(Conv 1*1 (F fuse )) Among them, σ is the Sigmoid function, the output range is [0,1], F fuse It is the sum of the up-sampled feature maps at different levels, and Conv represents 1*1 convolution; Use weighted cross entropy loss (WCE) to supervise the training and enhance the supervision weight of edge pixels: in, is the edge pixel weight, and the non-edge pixel weight is 1-β; i is the true edge label of the i-th pixel (0 for non-edge or 1 for edge), p i is the predicted edge probability of the i-th pixel, and N is the total number of pixels in the batch.

Images