A production material surface defect detection method based on edge bias semantic guidance network model

Abstract

This invention discloses a semantic segmentation detection method for surface defects of production materials based on an edge-biased semantically guided network model, belonging to the field of image processing and automated detection technology. The detection method includes the following steps: S01. The original image of the production material is processed sequentially by a multi-layer encoder using a ResUnet++ feature extraction network, which sequentially performs batch normalization and ReLU activation, a two-dimensional convolution module, a residual connection module, and a compression and excitation module. The final output is provided by an editor; S02. The final output of the encoder is processed by a dilated spatial pyramid pooling module; S03. The output processed by the dilated spatial pyramid pooling module is decoded in a decoder and an edge-guided attention module is introduced to achieve semantic segmentation detection of surface defects of the production material. The detection method of this invention exhibits outstanding performance in terms of edge sharpness and detail recovery on the surface of the production material.

Description

Technical Field

[0001] This invention belongs to the field of image processing and automated detection technology, and in particular relates to a semantic segmentation and detection method for surface defects of production materials based on the Edge-biased Semantic Guided Network (EGS-Net) model. Background Technology

[0002] In industrial production, the quality of molded parts is paramount. However, during the complex raw material processing, various defects can easily appear on the surface of the material. To prevent these defects from developing into irreversible damage, timely inspection of the material is essential.

[0003] Currently, surface defect detection in industrial production processes still relies primarily on traditional manual inspection. Specialized, universal inspection systems are particularly scarce, leading to low efficiency, high subjectivity, numerous blind spots causing missed detections, and extremely poor accessibility. Furthermore, existing automated detection methods based on traditional image processing or ordinary convolutional neural networks typically require training for specific materials and scenarios, resulting in poor generalization ability. When the material, defect type, or imaging conditions of products on a production line change, it is often necessary to recollect large amounts of data and retrain the model, which is costly and time-consuming. Moreover, for defects with strong background noise, low contrast, and complex textures, especially subtle edge defects, recognition accuracy is low. Therefore, developing a universal, transferable defect detection method that can quickly adapt to different inspection scenarios has significant practical application value. Summary of the Invention

[0004] This invention provides a semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model. It aims to solve the problems of current surface defect detection methods for production materials, which require training for specific materials and specific scenarios, have poor generalization ability, and have low recognition accuracy for defects with strong background noise, low contrast, complex textures, especially fine edge defects.

[0005] The technical solution of this invention is implemented as follows:

[0006] A semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model includes the following steps:

[0007] S01. The original image of the production material is processed by a multi-layer encoder through the feature extraction network of ResUnet++, which sequentially performs batch normalization and ReLU activation (Batch Normal. & ReLU) module, two-dimensional convolution (Conv2D) module, residual connection module, and compression and excitation module. The editor provides the final output.

[0008] S02. The final output of the encoder is processed by the Atrous Spatial Pyramid Pooling (ASPP) module, which allows the network to better understand the features of defects of different sizes in the image;

[0009] S03. The output processed by the hollow space pyramid pooling module is decoded in the decoder. A parallel dual-path structure is adopted at the skip connection of the decoder. The feature map of the skip connection is copied into two copies and input to the attention gate and the edge-guided attention module (EGA) respectively. The edge-guided attention module receives three inputs: the original image, the prediction map of the previous layer decoder and the skip connection features. After fusing them, it is added to the features of the main decoder through the residual connection to enhance the edge-guided features and realize the semantic segmentation and detection of surface defects of production materials.

[0010] In step S03, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and fuses them. Specifically, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and performs feature alignment. The aligned features are enhanced with boundary perception through a triple attention mechanism. The triple attention outputs are spliced, convolutionally compressed, and activated to obtain preliminary fused features. The features are further refined through spatial attention. Finally, the refined features and the skip connection features are fused through the inter-attention module (CBAM).

[0011] The triple attention refers to reverse attention, boundary attention, and high-frequency feature attention.

[0012] In this invention, the feature extraction network of ResUnet++ embeds an Edge-Guided Attention (EGA) module in the skip connections, which strengthens the edge and boundary information of surface defects of production materials during feature fusion, thereby improving the model's ability to locate and segment small and vague defects on the surface of production materials (such as cold-rolled steel strip).

[0013] In this invention, the overall network of the feature extraction network of ResUnet++ adopts an encoder-decoder structure, with ResUNet++ as the backbone network.

[0014] In step S01, the processing of the multi-layer encoder includes the following steps: First, ResUnet++ is trained using the preprocessed image and mask. Each layer of the ResUnet++ encoder consists of two 3×3 convolutional modules, two batch normalization and activation function processing modules, and a residual connection module. The input first undergoes convolution to extract features, then normalization is performed to reduce the internal skew offset of the feature map. After normalization, nonlinearity is introduced through the activation function, and then convolution is performed again for output. The output is added to the feature map passed by the residual connection starting from the input of the same layer. The feature map that absorbs the residual is then processed by the Squeeze & Excitation (SE) module to enhance the channel features before being output to the next layer. The multi-layer processing of the encoder is performed in sequence.

[0015] In step S01, the production material is a metal material, preferably cold-rolled steel strip.

[0016] In step S01, the processing method of the batch normalization and ReLU activation module and the two-dimensional convolution module specifically includes: the input first enters the batch normalization and ReLU activation module for normalization and activation, and then enters the two-dimensional convolution module composed of 3×3 convolution kernels. The two-dimensional convolution module consists of four layers of convolution kernels: an input layer, a first layer, a second layer, a third layer, and a fourth layer. The two-dimensional convolution module of the input layer consists of 64 3×3 convolution kernels, the two-dimensional convolution module of the second layer consists of 128 convolution kernels, the two-dimensional convolution module of the third layer consists of 256 3×3 convolution kernels, and the two-dimensional convolution module of the fourth layer consists of 512 3×3 convolution kernels.

[0017] In step S01, the i-th layer of the encoder processes the input as follows: first, normalization is performed; after activation using the ReLU activation function, the first activated output of the i-th layer is obtained, followed by two-dimensional convolution; then, a round of normalization, ReLU activation, and two-dimensional convolution is repeated; before the output, it is added to the residual starting at the input to absorb the residual and obtain the output of this layer; before being used as input to enter the next layer, compression and excitation module operations are performed.

[0018] Furthermore, the mathematical formula for the processing of input X by the i-th layer of the encoder is abstracted as follows:

[0019] The normalization specifically refers to: ;

[0020] in, Represents the normalization function. For the input tensor received by the encoder at layer i from the previous layer, B is the batch size of X, and H and W are the height and width, respectively. Its input channel number; let the pixel value at (b,c,i,j) of each channel be... The average pixel value along the spatial dimension (B×H×W) of each channel is The variance along the spatial dimension is The network learns that the scaling parameter of the current layer is The offset parameter for the current layer is β, and a small constant is set to prevent the denominator from being divided by zero. .

[0021] The first activation output of the i-th layer is obtained after activation using the ReLU activation function. Specifically:

[0022] ;

[0023] The ReLU activation function is defined as follows: .

[0024] The two-dimensional convolution process is performed by... The output is obtained by performing a two-dimensional convolution operation. Specifically: ;

[0025] in, Let be the weight matrix of the convolutional module in the first convolution of the current i-th layer. , dimension Where k is the size of the convolution kernel, It is the number of intermediate channels (set manually). This represents the number of channels in the input feature map of the i-th layer; specifically: when i=1 (i.e., the first layer), C1=3 (RGB input image); when i=2, C2=16 (number of output channels in the first layer); when i>2, ... It equals the number of output channels of the previous layer; It is the bias term of the first convolution in the i-th layer; It is the output feature of the first convolution.

[0026] The repeated rounds of normalization, ReLU activation, and 2D convolution specifically yield the following results: :

[0027] ;

[0028] in, It is the weight matrix of the second convolution in the i-th layer, which is a hyperparameter set manually or by machine learning; It is the bias term of the second convolution in the i-th layer, which is a hyperparameter set manually or by machine learning; It is the input of the second convolution (i.e., the output of the first convolution); It is the output of the second convolution, which has undergone a round of processing: "normalization → ReLU activation → two-dimensional convolution".

[0029] The process of adding the residual from the input to the output before it is output absorbs the residual and obtains the output of this layer. Specifically, this is the final... Before outputting, the residual from the input is added to absorb the residual to obtain the output of this layer. ;

[0030] The compression and excitation modules are operated before the data is input to the next layer; let the compression output of this layer be... ( The channel weights learned by the network in the activation layer are: ( The artificial reduction ratio is set to r, and the Sigmoid function is expressed as... The weights of the first fully connected layer in the incentive module are ( The weights of the second fully connected layer are... ( Output After compression, the result is (This means taking all; for 'd' in the formula, it means taking all batches and all channels), followed by the compressed version. The importance weight is obtained by determining the importance of entering the learning channel in the incentive module:

[0031]

[0032] Finally, the compression and excitation modules output the input for the next layer.

[0033] ;

[0034] After passing through four layers of encoders, the encoder obtains the final output.

[0035] In step S02, the hollow spatial pyramid pooling module is the bridge of the U-shaped network, connecting the encoder output layer and the decoder input layer. It consists of two 1×1 convolution modules, three 3×3 dilated convolution modules, and one global average pooling and upsampling module.

[0036] The process using the dilated spatial pyramid pooling module is as follows: the input feature map first enters four branches: a 1×1 convolution module, three 3×3 dilated convolution modules, and finally, the original input feature map is compressed into a scalar by a global average pooling module. Then, a 1×1 convolution is performed on this scalar, followed by bilinear interpolation for upsampling to obtain the output of the fifth branch. After obtaining five different outputs, the feature maps of these five outputs are concatenated along the channel dimension and then integrated through a 1×1 convolution module to obtain the final output of the dilated spatial pyramid pooling. This is significant for the segmentation of surface defects in industrial materials, as the size differences of surface defects in different types of metal materials are obvious. Learning the size features of defects undoubtedly makes the network's classification more accurate.

[0037] For the final output given by the encoder ( ), where B is The batch size, C is The number of channels, H and W are respectively Width and height. ASPP for input The mathematical abstraction of the processing flow is as follows:

[0038] (1) Let the number of output channels of a 1×1 convolution module be... The weight is ( The encoder's final output then enters a 1×1 convolutional module to obtain a local feature map.

[0039] ;

[0040] (2) Let the i-th ( The output of each dilated convolution module is The number of output channels is Weights are ( ), expansion rate Then, for the input feature map O, weights are respectively... The expansion rate is The three 3×3 dilation convolution operations are as follows:

[0041] ;

[0042] (3) Let global average pooling be represented as ( Upsampling is represented as Then, the input feature map O is upsampled using the average pooling method across the entire military to obtain the global feature map:

[0043] ;

[0044] (4) Let the concatenation operation along the channel dimension for multiple images be represented as follows: The five feature maps are then concatenated to obtain a multi-scale feature map:

[0045] ;

[0046] (5) The final concatenated feature map is processed by weights of... ( The network learns a comprehensive feature map through the processing of 1×1 convolutional modules.

[0047] .

[0048] In step S03, each layer of the decoder consists of an attention gate module, an upsampling module, a concatenate module, two normalization and activation function processing modules, two 3×3 convolution modules, a residual connection, and a skip connection.

[0049] The decoding process in the decoder is as follows: The feature maps from the input of the previous layer and those passed through the skip connections of the same layer encoder first enter the attention gate for feature filtering and selection. The attention gate uses the high-level semantic information of the decoder layer as a standard to judge the importance of each spatial region in the low-level semantic feature map passed through the skip connections of the same layer encoder, and outputs a weight matrix to weight the decoder feature map. After the decoder feature map is weighted, the high-level semantic feature map of the decoder is not fused with the low-level semantic feature map initially, but instead enters the upsampling module to reconcile its spatial resolution with the low-level semantic feature map. In this paper, transposed convolution is used for upsampling to restore the resolution of the feature map. Subsequently, in the concatenation module, the low-level semantic information and the high-level semantic information are fused and concatenated. Specifically, the high-level semantic feature map and the low-level semantic feature map are concatenated along the channel dimension, increasing the number of channels while maintaining the same spatial resolution, thus allowing the network to learn more features. The fused feature map is normalized and then nonlinearly introduced by the ReLU activation function. Finally, it is concatenated at the output of this layer with the feature map passed through the residual connection originating from the concatenation module.

[0050] The decoder filters the features transmitted by the encoder at each layer through attention gates and progressively recovers the spatial resolution. Let the first layer be... Layer input is ( ), skip connection features are ( ),in , ;

[0051] The handling of attention gates:

[0052]

[0053] Where g is an intermediate feature of the attention gate; For processing features from skip connections Attention weight matrix; For processing features from the encoder Attention weight matrix; The feature map is from the encoder skip connections; The feature map comes from the decoder; Attention weights; : sigmoid activation function; The attention weight matrix is ​​used to convert the fused features g into attention weights. This is the weighted feature map.

[0054] The processing of feature fusion and upsampling includes:

[0055] ;

[0056] ;

[0057] in, The resolution of the upsampled feature map is increased by a factor of 2 through nearest neighbor interpolation; This is an upsampling operation; The concatenated feature map has a dimension of 1. Where B is the batch size of X, which is a manually set hyperparameter;

[0058] This is a feature concatenation operation that connects two feature maps along the channel dimension.

[0059] The convolution and residual connection is as follows:

[0060] ;

[0061] in, The first layer convolution weights are used for dimensionality reduction and are hyperparameters that are set manually or through machine learning. These are the weights for the second convolutional layer, used in the final output, and are hyperparameters that can be set manually or through machine learning. For decoder number The output feature map of the layer; These are the projective convolution weights, used to adjust dimensions when channels are mismatched; For the first The final output of the layer is then processed through residual connections.

[0062] Finally, the decoder output is further integrated with multi-scale information through the hollow spatial pyramid pooling module, and then pixel-level classification is achieved through 1×1 convolution and Sigmoid activation function.

[0063] To address issues such as blurred images, irregular defect boundaries, and significant differences in defect size on industrial material surfaces during actual industrial production, this paper proposes a ResUnet++ network structure model. By introducing an edge attention mechanism (EGA) into its backbone extraction network and using a right-skewed semantic guidance path, the model's ability to withstand complex and blurred backgrounds and extract minute multi-scale features is improved. This ensures that the model can quickly and accurately identify defect features on material surfaces on high-speed production lines or other production environments.

[0064] To enhance the model's ability to perceive defect boundaries, an Edge-Guided Attention (EGA) module is introduced at the skip connections of the ResUnet++ decoder. This module employs a parallel dual-path structure: the feature maps from the skip connections are duplicated and input into the attention gate and the EGA module respectively; simultaneously, the EGA module also receives the original input image. After fusing the three-path features, the EGA adds them to the main decoder features through residual connections, achieving edge-guided feature enhancement.

[0065] The reasons for choosing residual connections to establish a link with the main path in this invention, rather than concatenation operations, are as follows: First, residual connections directly enhance the response of edge regions by amplifying existing feature activation values ​​without changing the feature channel dimensions and basic structure; second, concatenation operations multiply the feature channels, leading to a sharp increase in the number of subsequent convolution parameters and computational cost (up to nearly eight times in a four-layer decoder), exacerbating gradient instability; finally, consistent with the residual design philosophy of the ResUnet++ backbone network, this approach can more effectively complete detailed features and is easier to optimize. The core idea of ​​the EGA module after introducing semantic guidance is to use high-frequency edge information of the image to guide the model to focus on the boundary regions of defects.

[0066] The edge-guided attention module receives three inputs: the original image, the prediction map from the previous decoder, and skip connection features; where the original image... Preview of the front-end decoder and skip connection features ;

[0067] First, align the input channels and dimensions:

[0068] right Perform 1×1 convolution:

[0069] ;

[0070] in, E is the original input image (RGB three channels); E is the edge feature map after 3×3 convolution processing, with the number of channels adjusted to C. This is a 3×3 convolution operation that converts a 3-channel input into a C-channel output.

[0071] right Perform 1×1 convolution and bilinear upsampling:

[0072] ;

[0073] in: This is a bilinear interpolation upsampling operation.

[0074] Following Sigmoid normalization yields the result;

[0075] ;

[0076] in: This is the upsampled predicted probability map, with its size adjusted to (H,W); For the process The normalized prediction probability plot has a value range of [0,1]. for Activation function.

[0077] Aligned features enhance boundary awareness through a triple attention mechanism:

[0078] Reverse attention, enhancing the background area to reduce false negatives:

[0079]

[0080] in: : is a reverse attention feature, through Enhance the background area; For skip connection feature maps (from the corresponding layer of the encoder);

[0081] Boundary attention, using the Laplacian operator to sharpen the predicted boundary :

[0082] ;

[0083] in, For boundary attention features, the Laplacian operator is used to sharpen the predicted boundaries; This is the Laplacian operator, used to extract edge information from the predicted probability map.

[0084] High-frequency feature attention injects edge details into the original image:

[0085]

[0086] in, For high-frequency feature attention, use the original edge features Inject detailed information; ⊙ represents element-wise multiplication, also known as Hadamard product, which is element-wise multiplication. After multiplying corresponding elements of two matrices, the result is output to the same position in the new matrix.

[0087] The triple attention outputs are concatenated, convolutionally compressed, and activated to obtain preliminary fused features:

[0088]

[0089] in, For the feature concatenation operation, the three attention branches are concatenated along the channel dimension; For a 3×3 convolution operation, the 3C channels are compressed into a C channel; To initially fuse features, convolutional compression and ReLU activation are applied.

[0090] Further refine the features using spatial attention:

[0091] in, For 3×3 convolution operations, spatial attention weights are generated; The feature map is refined by spatial attention;

[0092] Finally, the refined features and skip connection features are fused by the CBAM module, the number of channels is adjusted by a 1×1 convolution, and the residual is added to the main path output:

[0093]

[0094] in, Convolutional block attention module further enhances features; : This is a 1×1 convolution operation; adjust the number of output channels. The output characteristics of the EGA module; The final output is added to the main path after residual join; For the decoder The original output of the layer.

[0095] This design allows EGA to remain lightweight while effectively guiding the model to focus on defect boundaries and easily overlooked areas.

[0096] Furthermore, the skip connection process of the model of this invention can be expressed as:

[0097] ;

[0098] in, This is the output feature map of the i-th layer decoder; For convolution operations, 1×1 convolution is used for channel adjustment; For the feature concatenation operation, two feature maps are connected along the channel dimension; For upsampling operations, the spatial resolution of the feature map is increased by 2 times; For the first The output of the layer decoder (lower resolution, richer semantic information); This is the Edge-Guided Attention module, used to enhance features; E is the edge feature map, which is the preprocessed edge information of the input image. The output feature map of the i-th layer encoder (as a skip connection); This is the predicted probability map for the i-th layer, used for attention calculation in the EGA module.

[0099] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0100] Compared to the original ResUNet++ model, although ResUNet++ achieves excellent performance in semantic segmentation tasks in medical scenarios by fusing residual connections, attention mechanisms, and dense connection blocks, it still suffers from problems such as blurred boundary localization and loss of detailed features when dealing with surface defects of industrial materials with complex edges and fine textures. To address these challenges, this paper makes a key improvement to the original ResUNet++ architecture by embedding Edge Guided Attention (EGA) modules at all SkipConnections.

[0101] Compared with existing technologies, the technical solution of this invention has the following advantages: This invention introduces an EGA module on top of ResUNet++, continuously enhancing and supplementing the defect edge information in the feature map of the ResUNet++ decoder, resulting in more accurate segmentation. The original ResUNet++ directly passes the low-level semantic feature map of the encoder to its skip connections, concatenating or adding it with the upsampled features of the decoder. While this design integrates features from different levels, its utilization of high-frequency information such as edges and contours in the low-level semantic feature map passed from the skip connections is indirect and insufficient. The edge-biased semantic guidance network model of this invention, by introducing the EGA module, explicitly fuses and calibrates the high-frequency edge features of the input image, the boundary of the predicted map from the previous decoder, and the background information provided by reverse attention. This design enables the model to actively "focus" on the edge regions of defects and re-examine the global model for any missed defects during the feature fusion stage, significantly enhancing the model's edge perception and detail preservation capabilities.

[0102] The introduction of the EGA module in ResUNet++ complements the attention mechanism of the ResUNet++ model. The original ResUNet++ primarily uses channel attention or spatial attention modules (such as SEBlock or CBAM), whose attention weights are derived from the feature maps themselves, lacking explicit semantic guidance. The EGA module introduced in this paper provides a semantically guided attention mechanism. It utilizes preliminary predictions (Pred) and input edge information (Edge) as prior knowledge to guide the feature fusion process. This attention mechanism based on external semantic information is more targeted and interpretable than general attention mechanisms that rely solely on their own features.

[0103] The EGA module introduced in this invention ensures the integrity of the original model and the independence of its additional components. The introduction of the EGA module does not disrupt the original residual structure and skip connections of ResUNet++; instead, it enhances and purifies the information they transmit. Through an attention mechanism, it suppresses background noise unrelated to defect edges in skip connections, amplifies useful detail features, and enables the decoder to receive higher-quality, more targeted feature information, thereby contributing to the generation of more accurate segmentation maps.

[0104] In summary, compared to the original ResUNet++, the proposed EGS-Net model, by embedding the EGA module, implements a more interpretable feature fusion strategy guided by edge information. This improvement directly addresses the core challenge of boundary accuracy in steel defect segmentation. Theoretical and subsequent experimental results demonstrate that this model effectively improves segmentation performance, especially in edge sharpness and detail recovery. Attached Figure Description

[0105] Figure 1 A schematic diagram of the ResUnet++ network structure according to an embodiment of the present invention;

[0106] Figure 2 A schematic diagram of the workflow of a single-layer encoder in an edge-biased semantic guidance network model according to an embodiment of the present invention;

[0107] Figure 3 A schematic diagram of the workflow of the ASPP module based on the edge bias semantic guidance network model in this embodiment of the invention;

[0108] Figure 4 A schematic diagram of the workflow of a single-layer decoder in an edge-biased semantic guidance network model according to an embodiment of the present invention;

[0109] Figure 5 A schematic diagram of the workflow of the semantically guided EGA module;

[0110] Figure 6 A comparison of the network structure before and after the introduction of semantic guidance paths and EGA modules;

[0111] Figure 7 The structural diagram of the edge-biased semantic guidance network model in this embodiment of the invention. Detailed Implementation

[0112] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0113] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, top, bottom, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0114] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0115] It should be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component.

[0116] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0117] Current methods for detecting surface defects in production materials suffer from limitations such as the need for training on specific materials and scenarios, poor generalization ability, and low recognition accuracy for defects with strong background noise, low contrast, complex textures, and especially fine edge defects. To address these issues, this invention proposes a semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic-guided network model.

[0118] Example 1

[0119] A semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model includes the following steps:

[0120] S01. The original image of the production material is processed by a multi-layer encoder through the feature extraction network of ResUnet++, which sequentially performs batch normalization and ReLU activation (Batch Normal. & ReLU) module, two-dimensional convolution (Conv2D) module, residual connection module, and compression and excitation module. The editor provides the final output.

[0121] S02. The final output of the encoder is processed by the Atrous Spatial Pyramid Pooling (ASPP) module, which allows the network to better understand the features of defects of different sizes in the image;

[0122] S03. The output processed by the hollow space pyramid pooling module is decoded in the decoder. A parallel dual-path structure is adopted at the skip connection of the decoder. The feature map of the skip connection is copied into two copies and input to the attention gate and the edge-guided attention module (EGA) respectively. The edge-guided attention module receives three inputs: the original image, the prediction map of the previous layer decoder and the skip connection features. After fusing them, it is added to the features of the main decoder through the residual connection to enhance the edge-guided features and realize the semantic segmentation and detection of surface defects of production materials.

[0123] In step S03, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and fuses them. Specifically, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and performs feature alignment. The aligned features are enhanced with boundary perception through a triple attention mechanism. The triple attention outputs are spliced, convolutionally compressed, and activated to obtain preliminary fused features. The features are further refined through spatial attention. Finally, the refined features and the skip connection features are fused through the inter-attention module (CBAM).

[0124] The triple attention refers to reverse attention, boundary attention, and high-frequency feature attention.

[0125] This invention proposes an enhanced segmentation network based on ResUNet++, which aims to improve the model's ability to locate and segment small and blurred defects on the surface of cold-rolled steel strip by introducing an Edge-Guided Attention (EGA) module. The overall network adopts an encoder-decoder structure, with ResUNet++ as the backbone network. Its core innovation lies in embedding the EGA module in the skip connections to enhance the edge and boundary information of defects during feature fusion. The overall structure of the ResUNet++ network is as follows. Figure 1 As shown.

[0126] ResUnet++ is first trained using preprocessed images and masks. Each layer of the ResUnet++ encoder consists of two 3×3 convolutional modules, two normalization and activation function processing modules, and one residual connection module. The input first undergoes convolution to extract features, then normalization reduces the internal oblique offset of the feature map. After normalization, non-linearity is introduced through an activation function, followed by another convolution. This output is added to the feature map passed from the residual connection at the input of that layer. The feature map absorbing the residual is then processed by the Squeeze & Excitation (SE module) to enhance channel features before being output to the next layer. A schematic diagram of the encoder workflow of a layer in the edge-biased semantic-guided network model is shown below. Figure 2 As shown.

[0127] Regarding the Batch Norm & ReLU module + Conv2D module:

[0128] The input first enters the Batch Norm & ReLU module for normalization and activation, and then enters the Conv2D module composed of 3×3 convolutional kernels. The Conv2D modules of the input layer are composed of 64 3×3 convolutional kernels, the Conv2D modules of the second layer are composed of 128 convolutional kernels, the Conv2D modules of the third layer are composed of 256 3×3 convolutional kernels, and the Conv2D modules of the fourth layer are composed of 512 3×3 convolutional kernels.

[0129] The abstract mathematical formula for the processing of input X by the i-th layer of the encoder is as follows:

[0130] 1. First, perform batch normalization:

[0131] in, Represents the normalization function. For the input tensor received by the encoder at layer i from the previous layer, B is the batch size of X, and H and W are the height and width, respectively. Its input channel number; let the pixel value at (b,c,i,j) of each channel be... The average pixel value along the spatial dimension (B×H×W) of each channel is The variance along the spatial dimension is The network learns that the scaling parameter of the current layer is The offset parameter for the current layer is β, and a small constant is set to prevent the denominator from being divided by zero. .

[0132] 2. After activation using the ReLU activation function, the first activation output of the i-th layer is obtained. Specifically:

[0133]

[0134] The ReLU activation function is defined as follows: .

[0135] 3. Let the weights of the convolutional module in the first convolution of the current i-th layer be set to... ( ), where k is the size of the convolutional module, This refers to the number of intermediate channels. This parameter needs to be set manually. The bias is set for the first convolution. , The output is obtained by performing a 2D convolution operation using Conv2D:

[0136]

[0137] in, Let be the weight matrix of the convolutional module in the first convolution of the current i-th layer. , dimension , where k is the size of the convolution kernel (different layers have different kernel sizes, which are hyperparameters set manually and used in machine learning). It is the number of intermediate channels (set manually). This represents the number of channels in the input feature map of the i-th layer; specifically: when i=1 (i.e., the first layer), C1=3 (RGB input image); when i=2, C2=16 (number of output channels in the first layer); when i>2, ... It equals the number of output channels of the previous layer; It is the bias term of the first convolution in the i-th layer (dimension is ). (These are hyperparameters that are set manually or through machine learning). It is the output feature of the first convolution (after BN and ReLU activation).

[0138] 4. This is followed by a round of repeated normalization, ReLU activation, and 2D convolution to obtain...

[0139] in, It is the weight matrix of the second convolution in the i-th layer, which is a hyperparameter set manually or by machine learning; It is the bias term of the second convolution in the i-th layer, which is a hyperparameter set manually or by machine learning; It is the input of the second convolution (i.e., the output of the first convolution); It is the output of the second convolution, which has undergone a round of processing: "normalization → ReLU activation → two-dimensional convolution".

[0140] 5. Finally Before outputting, the residual from the input is added to absorb the residual to obtain the output of this layer. ;

[0141] 6. Before being input to the next layer, perform compression and excitation operations. Let the compressed output of this layer be... ( The channel weights learned by the network in the activation layer are: ( The artificial reduction ratio is set to r, and the Sigmoid function is expressed as... The weights of the first fully connected layer in the incentive module are ( The weights of the second fully connected layer are... ( Output After compression, the result is (This means taking all; for 'd' in the formula, it means taking all batches and all channels), followed by the compressed version. The importance weight is obtained by determining the importance of entering the learning channel in the incentive module:

[0142] ;

[0143] Finally, the SE module outputs the input for the next layer;

[0144] .

[0145] After four encoder layers, the encoder obtains the final output. This final output, before entering the decoder, needs to be processed by the ASPP module to allow the network to better understand the features of defects of different sizes in the image. ASPP acts as a bridge in the U-shaped network, connecting the encoder output layer and the decoder input layer. It consists of two 1×1 convolutional modules, three 3×3 dilated convolutional modules, and one global average pooling and upsampling module. The input feature map first enters four branches: a 1×1 convolutional module, three 3×3 dilated convolutional modules, and finally, the original input feature map is compressed into a scalar by a global average pooling and upsampling module. Then, a 1×1 convolution is performed on this scalar, followed by bilinear interpolation for upsampling to obtain the output of the fifth branch. After obtaining five different outputs, the feature maps of these five outputs are concatenated along the channel dimension and then integrated by a 1×1 convolutional module to obtain the final ASPP output. This is significant for the task of segmenting surface defects in industrial materials, as the size differences of surface defects in different types of metals are substantial. Learning the size characteristics of defects undoubtedly makes the network's classification more accurate. A schematic diagram of the ASPP module workflow based on the edge-biased semantic-guided network model is shown below. Figure 3 As shown.

[0146] For the final output given by the encoder ( ), where B is The batch size is given by C, the number of channels is given by H, and the height is given by W. ASPP is the input... The mathematical abstraction of the processing flow is as follows:

[0147] 1. Let the number of output channels of a 1×1 convolutional module be... The weight is ( The encoder's final output then enters a 1×1 convolutional module to obtain a local feature map.

[0148] 2. Let the i-th ( The output of each dilated convolution module is The number of output channels is Weights are ( ), expansion rate Then, for the input feature map O, weights are respectively... The expansion rate is The three 3×3 dilation convolution operations are

[0149] 3. Let global average pooling be represented as... ( Upsampling is represented as Then, the input feature map O is upsampled using the average pooling method across the entire military to obtain the global feature map:

[0150]

[0151] 4. Let the concatenation operation along the channel dimension for multiple images be represented as follows: The five feature maps are then concatenated to obtain a multi-scale feature map.

[0152]

[0153] 5. Finally, the concatenated feature map is weighted by... ( The network learns a comprehensive feature map through 1×1 convolutional modules.

[0154] Each layer of the Resunet++ decoder consists of an Attention Gate module, an Upsampling module, a Concatenate module, two normalization and activation function processing modules, two 3×3 convolutional modules, a residual connection, and a skip connection. The feature maps from the input of the previous layer and those passed from the skip connections of the same layer's encoder first enter the Attention Gate for feature filtering and selection upon reaching the decoder. The Attention Gate uses the high-level semantic information of the decoder layer as a standard to judge the importance of each spatial region in the low-level semantic feature maps passed from the skip connections of the same layer's encoder, outputting a weight matrix to weight the decoder feature maps. After the decoder feature maps are weighted, the high-level semantic feature maps of the decoder are not immediately fused with the low-level semantic feature maps. Instead, they first enter the Upsampling module to reconstruct their spatial resolution against the low-level semantic feature maps. In this paper, transposed convolutions are used for upsampling to restore the resolution of the feature maps. Subsequently, in the Concatenate module, low-level and high-level semantic information are fused and concatenated. Specifically, the high-level and low-level semantic feature maps are concatenated along the channel dimension, increasing the number of channels while maintaining the same spatial resolution, thus allowing the network to learn more features. The fused feature map is normalized and then non-linearly introduced by the ReLU activation function. Finally, it is concatenated at the output of this layer with the feature map passed from the residual connection originating from the Concatenate module. A schematic diagram of the decoder workflow of the edge-biased semantic-guided network model is shown below. Figure 4 As shown.

[0155] Each layer of the decoder filters the features passed from the encoder through an attention gate and gradually restores the spatial resolution.

[0156] Let the first Layer input is ( ), skip connection features are ( ),in , :

[0157] 1. Attention Gate:

[0158]

[0159]

[0160] ;

[0161] Where g is the intermediate feature of the attention gate (which fuses low-level and high-level spatial information through a gating mechanism). For processing features from skip connections The attention weight matrix is ​​a hyperparameter that is set manually or by machine learning. For processing features from the encoder The attention weight matrix is ​​a hyperparameter that is set manually or by machine learning. The feature map (low resolution, high semantic information) comes from the skip connections of the encoder. The feature map comes from the decoder (high resolution, low semantic information). The attention weights are normalized to the [0,1] interval using the sigmoid function; The sigmoid activation function, mathematically defined as follows: The usage in the above formula is as follows: ; The attention weight matrix is ​​used to convert the fused features g into attention weights. The weighted feature map is then processed using attention weights. Recalibrating the original features involves setting hyperparameters manually or through machine learning.

[0162] 2. Feature fusion and upsampling:

[0163]

[0164]

[0165] in, The resolution of the upsampled feature map is increased by a factor of 2 through nearest neighbor interpolation; For upsampling operations, the nearest neighbor interpolation pattern is used. For a 2x upsampling, the target location ( The pixel value of f( is determined by the following formula: f( = f(x, y); Let the pixel coordinates in the original image be (x, y). When the upsampling factor is 2, the target coordinates can be mapped back to the source coordinates after dividing by 2 during interpolation. Since the pixel position indices are all integers, they also need to be rounded down, i.e. ,in This indicates a round-down operation, where the pixel value at the target coordinate is directly copied from the pixel value at the source coordinate during interpolation; The concatenated feature map has a dimension of 1. Where B is the batch size of X, which is a manually set hyperparameter;

[0166] This is a feature concatenation operation that connects two feature maps along the channel dimension.

[0167] 3. Convolution and Residual Connections:

[0168]

[0169] in, The first layer convolution weights are used for dimensionality reduction and are hyperparameters that are set manually or through machine learning. These are the weights for the second convolutional layer, used in the final output, and are hyperparameters that can be set manually or through machine learning. For the decoder The output feature map of the layer; These are the projective convolution weights, used to adjust dimensions when channels are mismatched; For the first The final output of the layer is then processed through residual connections.

[0170] Finally, the decoder output is further integrated with multi-scale information by the ASPP module, and then pixel-level classification is achieved through 1×1 convolution and Sigmoid activation function.

[0171] Regarding surface defect recognition, in order to address the problems of blurred images, irregular defect boundaries, and huge differences in defect size of industrial material surfaces in actual industrial production, we based our model on the ResUnet++ network structure. By introducing EGA (Edge Attention) into its backbone extraction network and improving the model's ability to resist interference from complex and blurred backgrounds and extract small multi-scale features through a right-skewed semantic guidance path, we ensure that the model can quickly and accurately identify the defect features of material surfaces on high-speed production lines or other production environments.

[0172] Regarding the Edge Attention (EGA) mechanism, to enhance the model's ability to perceive defect boundaries, an Edge-Guided Attention (EGA) module is introduced at the skip connections of the ResUnet++ decoder. This module employs a parallel dual-path structure: the feature maps of the skip connections are duplicated and input to the attention gate and the EGA module respectively; simultaneously, the EGA module also receives the original input image. After fusing the three-path features, the EGA adds them to the main decoder features through residual connections, achieving edge-guided feature enhancement.

[0173] The reasons for choosing residual connections to link with the main path instead of concat operations are as follows: First, residual connections directly enhance the response of edge regions by amplifying existing feature activation values ​​without changing the feature channel dimension and basic structure; second, concatenation operations multiply the feature channels, leading to a sharp increase in the number of subsequent convolution parameters and computational cost (up to nearly eight times in a four-layer decoder), exacerbating gradient instability; finally, consistent with the residual design philosophy of the ResUnet++ backbone network, this approach can more effectively complete detailed features and is easier to optimize. The core idea of ​​the semantically guided EGA module is to use high-frequency edge information of the image to guide the model to focus on the boundary regions of defects. The workflow diagram of the semantically guided EGA module is shown below. Figure 5 As shown.

[0174] The EGA module receives three inputs: the original image. Preview of the front-end decoder and skip connection features First, align the input channels and dimensions:

[0175] right Perform 1×1 convolution:

[0176] ;

[0177] in, E is the original input image (RGB three channels); E is the edge feature map after 3×3 convolution processing, with the number of channels adjusted to C. This is a 3×3 convolution operation that converts a 3-channel input into a C-channel output.

[0178] right Perform 1×1 convolution and bilinear upsampling:

[0179]

[0180] in: This is a bilinear interpolation upsampling operation; let the target interpolation position be ( Let the source pixel coordinates of the top-left corner of the target position be (x1, y1), the source pixel coordinates of the top-right corner of the target position be (x2, y1), the source pixel coordinates of the bottom-left corner of the target position be (x1, y2), and the source pixel coordinates of the bottom-right corner of the target position be (x2, y2). The decimal part in the x-direction is dx = x' – x1, and the decimal part in the y-direction is dy = y' – y1. Let f(x,y) represent the pixel value at position (x,y). Then, the pixel value at the target position is calculated by the following formula:

[0181] f(x', y') = (1 - dx)(1 - dy) × f(x1, y1) + dx(1 - dy) × f(x2, y1) +(1 - dx)dy × f(x1, y2) + dx × dy × f(x2, y2).

[0182] Following Sigmoid normalization, we get:

[0183] ;

[0184] in: This is the upsampled predicted probability map, with its size adjusted to (H,W); For the process The normalized prediction probability plot has a value range of [0,1]. for Activation function.

[0185] Aligned features enhance boundary awareness through a triple attention mechanism:

[0186] Reverse attention, enhancing the background area to reduce false negatives:

[0187]

[0188] in: : is a reverse attention feature, through Enhance the background area; For skip connection feature maps (from the corresponding layer of the encoder);

[0189] Boundary attention, using the Laplacian operator to sharpen the predicted boundary:

[0190] ;

[0191] in, For boundary attention features, the Laplacian operator is used to sharpen the predicted boundaries; The Laplacian operator is used to extract edge information from the predicted probability map; the Laplacian operation consists of four steps:

[0192] [1] Gaussian blur: For a pixel at coordinates (x, y), a Gaussian kernel is first generated for the corresponding range. Its mathematical form is:

[0193]

[0194] Where σ = 1.0; then the pixel value at position (i,j) Perform Gaussian smoothing.

[0195]

[0196] Where m, n∈[-2,2](5×5 Gaussian kernel);

[0197] [2] Downsampling: Pixel value after downsampling at position (x,y) ;

[0198] [3] Upsampling: First, the image size is doubled using nearest neighbor interpolation. Then perform Gaussian smoothing. ;

[0199] [4] Finally, the difference calculation is performed on the input image I of the Laplacian module: .

[0200] High-frequency feature attention injects edge details into the original image:

[0201] ;

[0202] in, For high-frequency feature attention, use the original edge features Inject detailed information; ⊙ represents element-wise multiplication, also known as Hadamard product, which is element-wise multiplication. After multiplying corresponding elements of two matrices, the result is output to the same position in the new matrix.

[0203] The triple attention outputs are concatenated, convolutionally compressed, and activated to obtain preliminary fused features:

[0204] in, For the feature concatenation operation, the three attention branches are concatenated along the channel dimension; For a 3×3 convolution operation, the 3C channels are compressed into a C channel; To initially fuse features, convolutional compression and ReLU activation are applied.

[0205] Further refine the features using spatial attention:

[0206] in, For 3×3 convolution operations, spatial attention weights are generated; The feature map is refined by spatial attention;

[0207] Finally, the refined features and skip connection features are fused by the CBAM module, the number of channels is adjusted by a 1×1 convolution, and the residual is added to the main path output:

[0208]

[0209] in, Convolutional block attention module further enhances features; : This is a 1×1 convolution operation; adjust the number of output channels. The output characteristics of the EGA module; The final output is added to the main path after residual join; For the decoder The original output of the layer.

[0210] This design allows EGA to remain lightweight while effectively guiding the model to focus on defect boundaries and easily overlooked areas.

[0211] Among them, the convolutional block attention module To further enhance features, the specific implementation method is as follows:

[0212] For the input image X, the mathematical form of the CBAM module is:

[0213] ;

[0214] After image X enters the CBAM module, it first passes through the channel attention mechanism, and then enters the spatial attention mechanism for processing:

[0215] (1). Channel attention mechanism:

[0216] [1]. Global pooling operation: Average pooling and max pooling are performed in parallel, with average pooling— Max pooling — ;in ;

[0217] [2]. Multilayer perceptron: The two results after pooling are first subjected to dimensionality reduction transformation. Then activated by the activation function. Finally, perform a dimensional transformation. ;in is the first-layer weight matrix, which is a hyperparameter set manually or by machine learning; r is the dimensionality reduction ratio, which is a hyperparameter set manually. It is the bias vector of the first layer, which is a hyperparameter set manually or by machine learning; similarly... and These represent the weight matrix and bias vector for the second time, respectively; max(x,0) is the ReLU activation function.

[0218] [3]. Channel attention weight calculation: The weights are summed by directly adding the images after dimensionality increase. Then sigmoid normalization was performed. Then the weights are expanded. Finally, the features of the original image are modulated using the expanded weights. ,in This represents the outer product operation. This indicates element-wise multiplication. A matrix of size H×W consisting entirely of 1s;

[0219] (2). Spatial attention mechanism:

[0220] [1]. First, perform channel aggregation: The input image first undergoes maximum aggregation and average aggregation operations in parallel. , , This represents the maximum value at each position, where This represents the average value at each location; then the features are concatenated. ;

[0221] [2]. Recalculate spatial attention weights: First perform a 7×7 spatial convolution. ,in The spatial convolution kernel is a hyperparameter that can be set manually or through machine learning; then the spatial attention weights are calculated. ;

[0222] [3]. Finally, feature modulation is performed: .

[0223] Compared to the original ResUNet++ model, although ResUNet++ achieves excellent performance in semantic segmentation tasks in medical scenarios by fusing residual connections, attention mechanisms, and dense connection blocks, it still suffers from problems such as blurred boundary localization and loss of detailed features when dealing with surface defects of industrial materials with complex edges and fine textures. To address these challenges, this paper makes a key improvement to the original ResUNet++ architecture by embedding Edge Guided Attention (EGA) modules at all SkipConnections. A comparison with the original model is provided below. Figure 6 As shown.

[0224] This paper introduces an EGA module on top of ResUNet++, continuously enhancing and supplementing defect edge information in the feature maps of the ResUNet++ decoder, resulting in more accurate segmentation. The original ResUNet++ directly passes the low-level semantic feature maps of the encoder to its skip connections, concatenating or adding them with the upsampled features of the decoder. While this design integrates features from different levels, its utilization of high-frequency information such as edges and contours from the low-level semantic feature maps passed through skip connections is indirect and insufficient. Our model (A-ResUNet++) introduces the EGA module to explicitly fuse and calibrate the high-frequency edge features of the input image, the boundaries of the predicted map from the previous decoder layer, and the background information provided by reverse attention. This design allows the model to actively "focus" on the edge regions of defects during the feature fusion stage and re-examine the global model for any missed defects, significantly enhancing the model's edge perception and detail preservation capabilities.

[0225] The introduction of the EGA module in ResUNet++ complements the attention mechanism of the ResUNet++ model. The original ResUNet++ primarily uses channel attention or spatial attention modules (such as SEBlock or CBAM), whose attention weights are derived from the feature maps themselves, lacking explicit semantic guidance. The EGA module introduced in this paper provides a semantically guided attention mechanism. It utilizes preliminary predictions (Pred) and input edge information (Edge) as prior knowledge to guide the feature fusion process. This attention mechanism based on external semantic information is more targeted and interpretable than general attention mechanisms that rely solely on their own features.

[0226] The EGA module introduced in this paper ensures the integrity of the original model and the independence of its additional components. The introduction of the EGA module does not disrupt the original residual structure and skip connections of ResUNet++; instead, it enhances and cleans the information they transmit. Through an attention mechanism, it suppresses background noise unrelated to defect edges in skip connections and amplifies useful detail features, enabling the decoder to receive higher-quality, more targeted feature information, thereby contributing to the generation of more accurate segmentation maps.

[0227] Let E be the edge features extracted from the input image. For the output of the i-th layer encoder, For the auxiliary prediction output at the corresponding scale during the decoding process, our mathematical extension comparison is as follows:

[0228] The skip connections in the original ResUNet++ can be simply represented as:

[0229]

[0230] The skip connection process in this model can be represented as:

[0231] .

[0232] in, This is the output feature map of the i-th layer decoder; For convolution operations, 1×1 convolution is used for channel adjustment; For the feature concatenation operation, two feature maps are connected along the channel dimension; For upsampling operations, the spatial resolution of the feature map is increased by 2 times; For the first The output of the layer decoder (lower resolution, richer semantic information); This is the Edge-Guided Attention module, used to enhance features; E is the edge feature map, which is the preprocessed edge information of the input image. The output feature map of the i-th layer encoder (as a skip connection); This is the predicted probability map for the i-th layer, used for attention calculation in the EGA module.

[0233] The structure diagram of the edge-biased semantic guidance network model in this embodiment of the invention is as follows: Figure 7 As shown.

[0234] In summary, compared to the original ResUNet++, the proposed EGS-Net model, by embedding an EGA module, implements a more interpretable feature fusion strategy guided by edge information. This improvement directly addresses the core challenge of boundary accuracy in steel defect segmentation. Theoretical and subsequent experimental results demonstrate that this model effectively improves segmentation performance, especially in edge sharpness and detail recovery.

[0235] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model, characterized in that: Includes the following steps: S01. The original image of the production material is processed by a multi-layer encoder consisting of batch normalization and ReLU activation module, two-dimensional convolution module, residual connection module, and compression and excitation module through the feature extraction network of ResUnet++, and the editor provides the final output. S02. The final output of the encoder is processed by the hollow spatial pyramid pooling module, which allows the network to better understand the features of defects of different sizes in the image; S03. The output processed by the hollow space pyramid pooling module is decoded in the decoder. A parallel dual-path structure is adopted at the skip connection of the decoder. The feature map of the skip connection is copied into two copies and input to the attention gate and the edge-guided attention module respectively. The edge-guided attention module receives three inputs: the original image, the prediction map of the previous layer decoder, and the skip connection features. After fusing them, it is added to the current layer features on the main path of the decoder through the residual connection to enhance the edge-guided features and realize the semantic segmentation and detection of surface defects of production materials.

2. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: In step S03, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and fuses them. Specifically, the edge-guided attention module receives three inputs—the original image, the prediction map of the previous layer decoder, and the skip connection features—and performs feature alignment. The aligned features are enhanced with a triple attention mechanism to enhance boundary perception. The triple attention outputs are spliced, convolutionally compressed, and activated to obtain preliminary fused features. The features are further refined through spatial attention. Finally, the refined features and the skip connection features are fused by the spatial attention module.

3. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 2, characterized in that: The triple attention refers to reverse attention, boundary attention, and high-frequency feature attention.

4. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: The overall network of the feature extraction network of ResUnet++ adopts an encoder-decoder structure, with ResUNet++ as the backbone network.

5. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: The processing of the multi-layer encoder includes the following steps: First, ResUnet++ is trained using the preprocessed image and mask. Each layer of the ResUnet++ encoder consists of two 3×3 convolutional modules, two batch normalization and activation function processing modules, and one residual connection module. The input first undergoes convolution to extract features, then normalization is performed to reduce the internal skew offset of the feature map. After normalization, nonlinearity is introduced through the activation function, followed by another convolution output. The output is added to the feature map passed by the residual connection starting from the input of the same layer. The feature map absorbing the residual is then compressed and activated to enhance the channel features before being output to the next layer. This multi-layer encoder processing is performed sequentially.

6. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: In step S01, the production material is cold-rolled steel strip.

7. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: In step S01, the processing procedure of the i-th layer of the encoder for the input is as follows: first, normalization is performed; after activation using the ReLU activation function, the first activated output of the i-th layer is obtained, followed by two-dimensional convolution; then, a round of repeated normalization, ReLU activation, and two-dimensional convolution is performed; before the output, it is first added to the residual starting at the input to absorb the residual and obtain the output of this layer. The compression and excitation modules are performed before the material is used as input to the next layer.

8. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: In step S02, the hollow spatial pyramid pooling module is the bridge of the U-shaped network, connecting the encoder output layer and the decoder input layer. It consists of two 1×1 convolution modules, three 3×3 dilated convolution modules, and one global average pooling module. The process using the dilated spatial pyramid pooling module is as follows: the input feature map first enters four branches: a 1×1 convolution module, three 3×3 dilated convolution modules, and finally, the original input feature map enters a global average pooling module to compress it into a scalar. Then, a 1×1 convolution is used to convolve this scalar, and then bilinear interpolation is used to upsample it to obtain the output of the fifth branch. After obtaining five different outputs, the feature maps of these five outputs are concatenated along the channel dimension and then integrated through a 1×1 convolution module to obtain the final output of the dilated spatial pyramid pooling.

9. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: In step S03, each layer of the decoder consists of an attention gate module, an upsampling module, a splicing module, two normalization and activation function processing modules, two 3×3 convolution modules, a residual connection, and a skip connection; The decoding process in the decoder is as follows: The feature maps from the input of the previous layer and those passed through the skip connections of the same layer encoder first enter the attention gate for feature filtering and selection. The attention gate uses the high-level semantic information of the decoder layer as a standard to judge the importance of each spatial region in the low-level semantic feature map passed through the skip connections of the same layer encoder, and outputs a weight matrix to weight the decoder feature map. After the decoder feature map is weighted, the high-level semantic feature map of the decoder is not fused with the low-level semantic feature map. Instead, it first enters the upsampling module and performs the following spatial resolution processing with the low-level semantic feature map: transposed convolution is used to upsample and restore the resolution of the feature map. Subsequently, in the concatenation module, the low-level semantic information and the high-level semantic information are fused and concatenated. Specifically, the high-level semantic feature map and the low-level semantic feature map are concatenated along the channel dimension, increasing the number of channels while maintaining the same spatial resolution, allowing the network to learn more features. The fused feature map is normalized and then nonlinearly introduced by the ReLU activation function. Finally, it is concatenated at the output of this layer with the feature map passed through the residual connection originating from the concatenation module.

10. The semantic segmentation and detection method for surface defects in production materials based on an edge-biased semantic guided network model according to claim 1, characterized in that: The skip connection process of the model is represented as follows: ； Among them, Decoder i The output feature map of the i-th layer decoder; Conv is a convolution operation, a 1×1 convolution used for channel adjustment; Concat is a feature concatenation operation, connecting two feature maps along the channel dimension; UpSample is an upsampling operation, increasing the spatial resolution of the feature map by a factor of 2; Decoder i+1 The output of the (i+1)th layer decoder; EGA is the Edge-Guided Attention module used to enhance features; E is the edge feature map, which is the preprocessed edge information of the input image; Encoder i P is the output feature map of the i-th layer encoder; i This is the predicted probability map for the i-th layer, used for attention calculation in the EGA module.

Images