A steel surface defect detection method fusing state space and attention

Abstract

The application discloses a steel surface defect detection method fusing state space and attention, and relates to the technical field of computer vision. The method comprises the following steps: acquiring a steel surface image and preprocessing; constructing an improved YOLOv8 detection model, realizing local-global double-flow feature extraction by improving a C2f_LS module combined with reparameterization convolution and large kernel convolution, performing lossless multi-scale context aggregation by using a MDCR module, and embedding a CAFMAttention module to enhance feature discrimination; designing a VSSFPN neck network, designing a VSSLite module based on a state space model, establishing global long-distance dependence by using a 2D selective scanning mechanism, and performing dynamic feature fusion; and finally outputting a detection result by decoupling a detection head. The application solves the problems of low detection precision of micro and low-contrast defects and difficulty in long-distance dependence modeling in the prior art under a complex industrial background, significantly improves the detection precision of steel surface defects while ensuring real-time performance.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and industrial inspection technology, and in particular to an improved method for detecting defects on steel surfaces that integrates state space and attention. Background Technology

[0002] In modern industrial manufacturing, steel, as a basic raw material, directly affects the performance and service life of the final product due to its surface quality. However, affected by rolling processes, mechanical damage, and environmental factors, steel surfaces inevitably develop various types of defects such as cracks, holes, scratches, and oxide scale. Traditional steel surface defect detection mainly relies on manual visual inspection. This method is not only labor-intensive and inefficient, but also highly susceptible to the subjective factors and fatigue of the inspectors, leading to missed or false detections. Although traditional machine vision-based inspection methods have achieved a certain degree of automation, their reliance on manually designed feature extraction operators often lacks sufficient robustness and generalization ability for complex and changing industrial environments and diverse defect types, making it difficult to meet the demands for high-precision, high-efficiency real-time detection.

[0003] In recent years, convolutional neural networks, represented by the YOLO series, have achieved remarkable results in the field of industrial object detection. However, existing CNN-based detection algorithms still face technical bottlenecks: on the one hand, limited by the local receptive field mechanism of the convolutional kernel, existing models struggle to effectively capture long-distance dependencies and global contextual information in images, resulting in insufficient detection capabilities for small defects or long cracks with slender features against a large-scale background; on the other hand, while introducing the Transformer architecture can enhance global modeling capabilities, its quadratic computational complexity leads to enormous computational costs, making real-time deployment on edge computing devices with limited computing power difficult. Furthermore, in the feature fusion stage, existing multi-scale fusion methods often struggle to effectively balance deep semantic information with shallow detail features, resulting in inaccurate defect feature extraction under complex background interference. Therefore, there is an urgent need to develop a steel surface defect detection method that can efficiently capture global visual information, maintain low computational costs, and possess strong cross-scale feature fusion capabilities. Summary of the Invention

[0004] This invention addresses the technical problems in existing steel surface defect detection tasks, such as the large defect scale span, low defect-background contrast, and the difficulty of traditional models in balancing long-distance dependency modeling and computational efficiency. It proposes a steel surface defect detection method based on local-global hybrid perception and bidirectional feature pyramid in state space.

[0005] To achieve the above objectives, the present invention provides the following technical solution.

[0006] In a first aspect, embodiments of the present invention provide a method for detecting surface defects in steel that integrates state space and attention, comprising the following steps:

[0007] Step S1: Obtain an image of the steel surface to be detected, and perform uniform resolution adjustment and mosaic enhancement preprocessing on the image to obtain an input feature map;

[0008] Step S2: Input the input feature map into the local-global hybrid perception backbone network for hierarchical feature extraction; the backbone network generates a multi-scale feature map set containing low- and mid-level and high-level features by synergistically using the local-global dual-stream feature extraction C2f_LS module, the multi-dilation channel refinement MDCR module, and the cross-axis fusion modulation attention module.

[0009] Step S3: Input the multi-scale feature mapping set into the state-space bidirectional feature pyramid neck network VSSFPN for dynamic fusion; the neck network is based on a bidirectional weighted topology and uses a lightweight visual state space module to perform state-space dynamic filtering and recalibration on the feature stream, and outputs the fused high-level semantics and shallow detail features.

[0010] Step S4: Input the fused features into the decoupled detection head, extract spatial features and category features using a shared convolutional layer, and perform positive and negative sample matching through a dynamic task alignment allocator to finally output the category and bounding box position of the steel surface defect.

[0011] Furthermore, the local-global dual-stream feature extraction module C2f_LS in step S2 employs a dual-stream parallel feature decoupling mechanism, specifically including:

[0012] Step S21, First branch local flow: Using reparameterized convolution RepVGGDW combined with efficient channel attention ECAAttention, extract the fine defect texture and channel selectivity features of the image;

[0013] Step S22, Second Branch Global Flow: The large kernel convolution strategy LSConv is adopted, which consists of large kernel perception and small kernel aggregation; wherein the large kernel perception uses a 7×7 large-scale convolution kernel to extract broad background information and generate dynamic weights to expand the effective receptive field;

[0014] Step S23, Feature Fusion: The output features of the first branch and the second branch are concatenated and fused through 1×1 convolution to achieve dynamic complementarity between local texture details and global semantic recognition capabilities.

[0015] Furthermore, in step S2, the multi-dilation channel refinement module replaces the traditional spatial pyramid pooling to achieve lossless multi-scale context aggregation; the MDCR module adopts a multi-branch parallel structure, and each branch processes the feature map through depth-separable dilated convolution with different dilation rates, covering multiple receptive field intervals without reducing spatial resolution, thereby fully preserving the spatial localization information and edge details of minor defects while aggregating global context information.

[0016] Step S7, further, in step S2, the cross-axis fusion modulation attention module is located in the backbone network output stage and is used to jointly refine the features of the channel and spatial dimensions. The module generates channel attention maps and spatial attention maps through a two-level gating mechanism, and modulates the features through cross-axis multiplication operations to adaptively enhance the feature response of the defect area and suppress background noise interference such as rolling texture.

[0017] Furthermore, in step S3, the state-space bidirectional feature pyramid neck network includes a top-down path and a bottom-up path;

[0018] Step S31: In the top-down path, high-level features are upsampled and then fused with mid-level features through a weighted summation module;

[0019] Step S32: In the bottom-up path, the low-level features are downsampled and then fused with the backbone network features and the uplink path features;

[0020] Step S33: The fused features are refined at each node using a lightweight visual state space module to establish global dependencies between pixels on the feature map.

[0021] Furthermore, the lightweight visual state space module includes an input projection layer, three parallel processing branches, and an output fusion layer, and its specific processing procedure is as follows:

[0022] Step S331: The input features first pass through a projection layer consisting of 1×1 convolution, batch normalization, and SiLu activation function;

[0023] Step S332: The projected features simultaneously enter three parallel branches: the Mamba state space branch, the local convolution branch, and the shortcut branch.

[0024] Step S333: The state space branch of the state space model is used to capture long-distance spatial dependencies. Specific steps include:

[0025] After layer normalization, the features are entered into the SS2D selective scanning unit;

[0026] Sequence unpacking: SS2D unpacks two-dimensional image features into a one-dimensional sequence along four different directions: top to bottom, bottom to top, left to right, and right to left.

[0027] Selective scanning: The expanded sequence is processed in parallel using S6 blocks, and historical information is passed and remembered through compressed state vectors;

[0028] Cross-merging: The processed sequences are restored and merged into a two-dimensional feature map, and global context information is established with linear computational complexity.

[0029] Step S334: The local convolution branch is used to extract local texture details. The specific steps include: using 3×3 depthwise separable convolution to extract spatial features independently for each channel, and then using 1×1 pointwise convolution to perform information interaction between channels.

[0030] Step S335: The output fusion layer performs the following operations: the output features of the three parallel branches are aggregated element by element; then the weight vector between channels is learned through the SE channel attention module, and the aggregated features are adaptively weighted; finally, the 1×1 convolutional layer is projected back to the original dimension, and residual connections are made with the initial input of the module to obtain the final output.

[0031] Furthermore, in step S4, the loss function of the detection head uses binary cross-entropy loss to calculate the classification error, and combines distributed focus loss and perfect intersection-union ratio loss to calculate the regression error. The detection accuracy and convergence stability are optimized through weighted balancing.

[0032] Secondly, embodiments of the present invention also provide a steel surface defect detection system, comprising:

[0033] Image acquisition module, used to acquire raw image data of the steel surface;

[0034] The preprocessing module is used to unify and enhance the resolution of the image;

[0035] The feature extraction module, which includes C2f_LS, MDCR and CAFMAttention sub-modules, is used to extract multi-scale local-global mixed features of images.

[0036] The feature fusion module, which includes the VSSFPN and VSSLite sub-modules, is used to perform state space-based dynamic fusion and context alignment of multi-scale features.

[0037] The defect detection module is used to output the category and location information of defects based on fused features.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] 1. Strong local-global hybrid perception capability: This invention combines the local texture extraction capability of RepVGG with the large kernel global perception capability of LSConv by designing the C2f_LS module in the backbone network, effectively solving the problem of extremely diverse scale of steel defects. It can accurately locate tiny pinholes and completely identify large-area corrosion.

[0040] 2. Lossless multi-scale context aggregation: The proposed MDCR module uses multi-dilation rate dilated convolution to replace the traditional pooling operation, avoiding the loss of spatial information during downsampling and significantly improving the edge localization accuracy for low contrast and small defects.

[0041] 3. Efficient long-distance dependency modeling: The VSSLite module is introduced in the feature fusion stage. By utilizing the linear complexity of the Mamba state space model, a global receptive field similar to that of the Transformer is achieved without significantly increasing the computational burden. This effectively solves the problem of insufficient long-distance contextual information modeling in complex industrial contexts.

[0042] 4. Improved robustness and feature discrimination: Through cross-axis modulation of CAFMAttention and dynamic feature propagation of VSSFPN, the model can adaptively suppress background noise such as rolling texture, and exhibits excellent robustness in low-contrast defect detection tasks.

[0043] 5. The introduction of a heavy-parameter lightweight detection head helps reduce the complexity of the model and the model's demand for computing resources, enabling the method to be deployed and applied on a wider range of hardware platforms. Attached Figure Description

[0044] Figure 1 This is a schematic flowchart of a method for detecting surface defects in steel provided by some embodiments of this application;

[0045] Figure 2 This is a structural diagram of an improved YOLOv8 network model for a steel surface defect detection method provided in some embodiments of this application;

[0046] Figure 3 This is a C2F-LS module structure diagram of a steel surface defect detection method provided in some embodiments of this application;

[0047] Figure 4 This is a VSSFPN network structure diagram of a steel surface defect detection method provided in some embodiments of this application;

[0048] Figure 5 This is a VSSLite module structure diagram of a steel surface defect detection method provided in some embodiments of this application;

[0049] Figure 6 This is a structural diagram of the MDCR module of a steel surface defect detection method provided in some embodiments of this application;

[0050] Figure 7 This is a structural diagram of the CAFMAttention module of a steel surface defect detection method provided in some embodiments of this application;

[0051] Figure 8 This is a comparison chart of evaluation indicators before and after the implementation of a steel surface defect detection method provided by some embodiments of this application;

[0052] Figure 9 These are detection effect diagrams of a steel surface defect detection method provided by some embodiments of this application; Detailed Implementation

[0053] The restraint scheme of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention provides a method for detecting surface defects in steel, such as... Figure 1 As shown, the method includes the following steps:

[0055] Step S1: Obtain the NEU-DET dataset of steel surface defects and preprocess the dataset to obtain the dataset for training.

[0056] Step S2: Using YOLOv8 as the base model with a model size of n, the improved YOLOv8 network backbone network replaces the Bottleneck in C2f with an improved DS-LSBlock block to form the C2f-LS block; and replaces the SPPF block in the traditional network with an MDCR block; and embeds a CAFMAttention module at the backbone output to jointly refine the channel and spatial features.

[0057] Step S3: Construct a state-space-based bidirectional feature pyramid neck network. Specifically, this includes: constructing a top-down and bottom-up bidirectional fusion path using a weighted topology structure of BiFPN; introducing a lightweight visual state-space module VSSLite at the feature fusion node, wherein the VSSLite module integrates a 2D selective scanning unit SS2D, and uses the state-space model to perform dynamic filtering and long-distance dependency modeling on multi-scale features to generate a fusion feature map with global context consistency;

[0058] Step S4: Input the target dataset images obtained in step S1 into the VLS-YOLO network model constructed in steps S2 and S3, such as... Figure 2 As shown. Initial training parameters are set, features are predicted using a decoupled detection head, and the total loss is calculated based on binary cross-entropy loss, distribution focus loss, and complete intersection-union ratio loss. The model parameters are updated through backpropagation to train a deep learning model for steel surface defect detection.

[0059] Step S5: Input the image of the steel surface to be inspected into the model obtained in step S4 for inspection. The model outputs the inspection results including the probability of the defect category and the coordinates of the bounding box, so as to achieve accurate positioning of multi-scale defects such as pinholes, scratches, and corrosion.

[0060] The above steps will be explained in detail below with reference to specific content:

[0061] The specific implementation method of step S1 includes the following steps:

[0062] Step S11: Obtain the NEU-DET dataset for steel surface defect detection and convert the acquired images to the desired format. The final dataset images are in YOLO format.

[0063] Step S12: Divide the YOLO format dataset images into training set, validation set and test set in an 8:1:1 ratio;

[0064] Step S13: Apply the Mosaic data augmentation method to the training set to obtain the augmented training set, and adjust the resolution of the augmented images to 512×512.

[0065] The improved YOLOv8 backbone network structure in step S2 is constructed using the following method:

[0066] Step S21: Construct the C2f_LS module to replace the original C2f module in the backbone network, such as... Figure 3 As shown; the C2f_LS module is constructed through a dual-stream decoupling mechanism, wherein the local stream branch uses RepVGGDW and ECAA attention concatenation to extract texture details, and the global stream branch uses large kernel convolution combined with large kernel perception and small kernel aggregation strategy to extract wide-area background information;

[0067] Step S22: Replace the original SPPF module at the end of the backbone with a multi-expansion channel refining module, such as... Figure 6 As shown; the MDCR module is constructed through parallel depthwise separable dilated convolutions, with each branch having a different dilation rate to cover multiple levels of receptive fields, and finally lossless context aggregation is achieved through channel splicing;

[0068] Step S23: Introduce a cross-axis fusion modulation attention module at the output of a specific feature layer of the backbone network, such as... Figure 7 As shown, this module generates attention weights through channel gating and spatial gating, and performs cross-axis multiplication modulation to enhance the discriminative power of the features.

[0069] The state-space bidirectional feature pyramid neck network in step S3 is constructed using the following method:

[0070] Step S31: Construct a bidirectional weighted feature pyramid structure based on the BiFPN topology, as follows: Figure 4 As shown, it includes a top-down upsampling fusion path and a bottom-up downsampling fusion path, using weighted summation to fuse feature flows at different scales;

[0071] Step S32: Construct a lightweight visual state space module to replace the feature extraction module in the neck network, such as... Figure 5 As shown; the VSSLite module adopts a three-branch parallel structure: the first branch is an SS2D unit based on the state space model, used to establish global long-distance dependencies; the second branch is a 3×3 depthwise separable convolution, used to extract local features; the third branch is a shortcut connection;

[0072] Step S33, the specific implementation of the SS2D unit is as follows: the input feature map is unfolded into a one-dimensional sequence along four directions, the sequence is selectively scanned and the state is updated using the S6 block, and finally it is restored into a two-dimensional feature map through cross-merging;

[0073] Step S34: Aggregate the output features of the three branches, recalibrate the features through the SE channel attention module, and finally output a high-quality fused feature map through residual connection.

[0074] Furthermore, the format conversion in step S11 is to convert the XML format annotation file into a TXT format annotation file that can be used in YOLO.

[0075] Furthermore, the Mosaic data augmentation method in step S13 refers to randomly selecting sixteen images from different scenes and stitching them together into a new synthetic image. This process first crops each image to the same size and then stitches them together in a 4x4 layout to form an image containing diverse backgrounds and targets.

[0076] The above details the specific preprocessing methods for the dataset.

[0077] Furthermore, the C2f_LS method constructed in step S21 includes the following steps:

[0078] For input tensor First through a The convolutional layer includes batch normalization and SiLU activation function for channel adjustment, outputting a tensor. Then The segments are divided along the channel dimension, resulting in two parts. and .in Keep it directly. The data is fed into n cascaded DS-LSBlock modules for deep feature extraction.

[0079] The DS-LSBlock block employs a dual-stream parallel architecture, simultaneously processing the input feature X within the module through both a local perception branch and a global perception branch.

[0080] Branch A, the local perception branch, is primarily used to extract texture edge features of subtle defects. It first extracts features through reparameterized convolutional blocks, then performs channel weighting using an ECA attention module, and finally transforms the data using a feedforward neural network (FFN). The RepVGGDW training phase includes... convolution, The three branches of convolution, identity mapping, and inference are fused into a single branch during the inference phase through structural reparameterization. Convolution, its output It can be represented as:

[0081]

[0082] in, For reparameterized convolution output, The feature map tensor input to this branch, For reparameterized Convolution kernel weight matrix, This is the reparameterized bias vector;

[0083] The ECA attention module aggregates spatial information through global average pooling and captures cross-channel interaction information using 1D convolution. Its calculation formula is as follows:

[0084]

[0085]

[0086] Wherein, GAP represents global average pooling. This represents a one-dimensional convolution with a kernel size of k. It is the Sigmoid activation function. This indicates element-wise multiplication. For the generated channel attention weight vector, For the forward feedback neural network transformation, This is to output the enhanced feature map of the final output of the local perceptual branch.

[0087] Branch B, the Global Perception Branch, is primarily used to capture wide-area background information using a large receptive field. This branch includes the LSConv module and FFN. LSConv consists of two parts: large kernel perception and small kernel aggregation. First, LKP uses a large-scale convolutional kernel decomposition strategy to extract spatial background features and generate dynamically selected weights. To reduce computational cost, LKP decomposes the large kernel convolution into a depthwise convolution sequence:

[0088]

[0089] in The convolution kernel is Deep convolutions provide a large effective receptive field. Let X be the generated attention weight tensor, and let X be the feature map input to this branch. for Convolution operation, This is a normalization operation.

[0090] Subsequently, SKA utilizes the generated weights Adaptive aggregation of input features to output features :

[0091]

[0092] in This is a small-core aggregation operation. For batch normalization operations, To output a feature map with wide-area background information from the global perception branch;

[0093] The output of the final branch B is .

[0094] in For the forward feedback neural network transformation, The final output of the global perception branch is the enhanced feature map.

[0095] Feature fusion: The output of branch A and the output of branch B The splicing is performed along the channel dimension, and through a... The convolutional layers are fused together, and finally a residual connection is made with the input X of the module to obtain the final output of DS-LSBlock. :

[0096]

[0097] Where X is the feature map input to this branch. for Convolution operation, For channel fusion operation, To output the enhanced feature map of the final output of the local perceptual branch, The final output of the global perception branch is the enhanced feature map. The outputs of branch A and branch B are concatenated and merged;

[0098] Finally, , The output features processed by all DS-LSBlock modules are concatenated along the channel dimension and then processed through a... The convolutional layers are aggregated to obtain the final output of the C2f_LS module.

[0099] The C2f_LS module employs a parallel "local-global" dual-stream design. Branch A utilizes RepVGG and ECA to focus on capturing high-frequency texture details, while Branch B leverages the large-kernel mechanism of LSConv to focus on modeling low-frequency background semantics. This decoupled design effectively solves the problem of simultaneously handling minute scratches and large-area patches in steel surface defect detection. Furthermore, the reparameterization characteristics of RepVGG and the depth decomposition strategy of LKP ensure efficient model inference.

[0100] Furthermore, the method for constructing the MDCR in step S22 includes the following steps:

[0101] For feature blocks: input feature tensors It is uniformly divided into 4 sub-tensors along the channel dimension. The number of channels for each subtensor is ;

[0102] For multi-scale hole detection: four sub-tensors are fed into parallel depthwise separable convolution branches for processing. To capture contextual information at different scales, the four branches are each configured with a dilation rate of [missing information]. The output of the i-th branch The calculation formula is:

[0103]

[0104] in Indicates the expansion rate Depthwise convolutions include convolution, normalization, and activation functions;

[0105] Cross-channel interaction: To facilitate fine-grained interaction between different receptive field features, to Perform channel-by-channel hybrid encoding. For each channel index... Extract the first branch corresponding to all branches The feature maps of each channel are stitched together, and then... convolution To merge:

[0106]

[0107] in This represents the feature of the j-th channel output from the i-th branch. For 1×1 convolution, This is for channel splicing operations. The features of the j-th channel group after fusion;

[0108] Finally, all the interactively processed channel features Reassembled to restore the complete tensor, and then passed through a... convolution The final feature refinement is performed to obtain the module output. :

[0109]

[0110] in The features after fusion of the j-th channel group, For 1×1 convolution, This is the multi-scale analysis text aggregation feature map that is the final output of the MDCR module;

[0111] The MDCR module, through its segmentation-transformation-interaction-aggregation design, utilizes multiple dilation rates to cover a broad receptive field from local details to the global background without reducing spatial resolution, thus achieving lossless contextual feature extraction.

[0112] Furthermore, the method for constructing the CAFMAttention block in step S23 includes the following steps:

[0113] Feature projection and enhancement: After expanding the dimensions of the input features, linear projection is performed through convolution, followed by local feature enhancement through depthwise convolution, generating a hybrid feature tensor QKV containing query, key, and value information;

[0114] Constructing local convolutional branches: Features are extracted from QKV, and the head dimension and channel dimension interact through dimensional rearrangement and deformation. Fully connected (FC) layers are used to aggregate multi-head features, followed by grouped depthwise convolutions to extract local spatial and channel dependencies, resulting in locally modulated features. :

[0115]

[0116] Where Q, K, and V are the query, key, and value tensors obtained from the input feature mapping, respectively. For dimensional reshaping operations, For fully connected layer operations, For grouped depthwise convolution operations, Features extracted from local convolutional branches;

[0117] Constructing a global self-attention branch: The QKV tensor is split into a query tensor Q, a key tensor K, and a value tensor V. After normalizing Q and K, a cross-axis attention map is calculated. A learnable temperature parameter is introduced into the calculation of the attention score. To control the smoothness of the distribution:

[0118]

[0119] Where Q, K, and V are the query, key, and value tensors obtained from the input feature mapping, respectively. This is the transpose of the key tensor. For learnable temperature parameters, For normalized exponential functions, The calculated global attention map;

[0120] Subsequently, V is weighted and aggregated using an attention map, and global modulation features are obtained through the output projection layer. :

[0121]

[0122] in For the convolution weights of the output projection layer, Features extracted for the global attention branch The calculated global attention map;

[0123] Cross-axis feature fusion: outputting local convolutional branches With global self-attention branch output Perform element-wise addition to output the final enhanced feature:

[0124]

[0125] in Features extracted for the global attention branch Features extracted from local convolutional branches The feature map output by the cross-axis fusion modulation attention mechanism module;

[0126] The CAFMAttention module effectively enhances the model's ability to discriminate low-contrast defects on steel surfaces by processing local high-frequency information and global low-frequency information in parallel and introducing a temperature coefficient-adjusted attention mechanism.

[0127] Furthermore, the method for constructing VSSFPN in step S31 is as follows:

[0128] This invention utilizes channel adjustment to perform feature maps of three different scales output from the backbone network. The convolutional layer uniformly adjusts the number of channels in the input neck network feature maps to 256. Subsequently, a bidirectional feature fusion path, including top-down and bottom-up approaches, is constructed. At the fusion nodes, a learnable weighted addition mechanism replaces simple element-wise addition, as shown in the following formula:

[0129]

[0130] in For input features, For learnable weights, To prevent the minimum value of division by zero, The output feature map after fusion;

[0131] The fused feature map is then processed After convolution, the data is fed into the VSSLite module for dynamic state-space filtering and feature refinement. This network structure combines the weighted topology of BiFPN with the long-range modeling capability of the state-space model. It can not only adaptively balance the feature contributions of different resolutions, but also establish the consistency of the global context in the feature pyramid, significantly improving the model's ability to detect low-contrast and multi-scale steel defects.

[0132] Furthermore, the method of the VSSLite module in step S32 is as follows:

[0133] The VSSLite module replaces the convolutional processing module in the original YOLO neck network. The VSSLite module is internally designed with a three-branch parallel structure to balance global and local information.

[0134] The first branch, the global branch, is where the input features, after being normalized by the layer, enter the SS2D unit based on the state space model to capture long-distance dependencies in the image.

[0135] Second branch local branch: using Depth-separable convolutions are used to extract local spatial textures, which are then processed... Convolution enables channel interaction, compensating for the shortcomings of state-space models in capturing high-frequency details;

[0136] The third branch, the shortcut branch, constructs an identity mapping path to directly pass the original input features, ensuring effective backpropagation of the gradient.

[0137] Furthermore, the specific implementation method of the SS2D unit in step S33 is as follows:

[0138] First, the input features are processed... After convolutional projection, batch normalization, and SiLU activation, the input sequence is fed into the SS2D unit via layer normalization. SS2D uses a four-directional scanning strategy to unfold the two-dimensional feature map into a one-dimensional sequence and employs the S6 block selective scanning mechanism for feature processing. For the input sequence... The discretized state-space model formula is as follows:

[0139]

[0140]

[0141] in, , . For time scale parameters, B is the state transition matrix, and B and C are the projection matrices. These are the state transition parameters after discretization. These are the discretized input parameters. Let be the hidden state variable of the previous time step t-1. Let be the hidden state variable at time t, where t is the time step index of the sequence. Output the sequence elements at time t;

[0142] By introducing parameters that change dynamically with the input data B, C, and SS2D can establish global dependencies between any two pixels on the feature map while maintaining linear computational complexity, effectively capturing long-distance defect features (such as long cracks) on the surface of steel.

[0143] Furthermore, the method for constructing the local convolutional branch of the VSSLite module in step S33 is as follows:

[0144] To compensate for SS2D's shortcomings in processing high-frequency detail information, a lightweight local convolutional branch is constructed in parallel. This branch directly receives the projected features and undergoes the following operations:

[0145]

[0146] in The feature tensor after input projection. Depth-separable convolutions with the number of groups equal to the number of channels are used to extract local spatial textures. For activation functions; This is pointwise convolution, used for information exchange between channels. For batch normalization operations, for Pointwise convolution operation, The output features of the local convolutional branch;

[0147] This branch enhances the model's ability to capture edge information of minute defects (such as pinholes and scratches) with extremely low computational cost.

[0148] Furthermore, the method for feature aggregation and SE recalibration in step S34 is as follows:

[0149]

[0150] in The input feature tensor is obtained by aggregating the outputs of the three branches. To enable adaptive global average pooling, These are the weight parameters of the first fully connected layer. The SiLU activation function is used. These are the weight parameters for the second fully connected layer. It is the Sigmoid activation function. The feature map is after channel attention recalibration;

[0151] The output features of the SS2D branch, the local convolution branch, and the shortcut branch are summed element-wise to obtain the aggregated features. To further enhance the discriminative power of features, an SE channel attention module is introduced: first, spatial information is compressed through global average pooling; then, the dependency relationship between channels is learned through two fully connected layers to generate a weight vector. Recalibrate. Finally, pass the recalibrated features through a... A convolutional projection layer is introduced, and an external residual connection is added to the initial input of the VSSLite module to output a high-quality fused feature map.

[0152] Furthermore, the network training in step S4 includes the following steps:

[0153] Model training hyperparameters were set as follows: 300 training cycles, initial learning rate of 0.01, weight decay of 0.0005, SGD optimizer, batch size of 10, input image size of 512×512, momentum of 0.937, Mosaic data augmentation was set to Boolean type and turned off in the last 10 cycles.

[0154] The training dataset images obtained in step S1 are input into the improved YOLOv8 network obtained in step S4 for training. CIoU is used to calculate the bounding box regression loss, binary cross-entropy loss is used to calculate the target confidence loss, and multi-class cross-entropy loss is used to calculate the classification loss. Backpropagation is then performed to update the weight parameters. Through training, a deep learning model for detecting defects on steel surfaces is obtained.

[0155] To evaluate model performance, evaluation metrics such as mean accuracy (mAP), number of parameters, and computational cost are introduced. The method for calculating mean accuracy is as follows:

[0156]

[0157] Where n represents the number of categories. Let AP represent the accuracy of the i-th category, P represent the accuracy rate, and mAP represent the average precision.

[0158] The formulas for calculating P and R are as follows:

[0159]

[0160]

[0161] Where T / F indicates whether the true / false prediction is correct, P / N indicates whether the prediction is positive or negative, P is precision, and R is recall.

[0162] The number of parameters in a model refers to the total number of learnable parameters in the network, including weights and biases. The number of parameters reflects the complexity and expressive power of the model.

[0163] Computational cost represents the computational resources required by the model when performing sequential forward inference; it reflects the computational efficiency of the model during inference or training.

[0164] Furthermore, in step S5: the improved YOLOv8 network model is used to perform object detection on the test set, generating images containing detection boxes, and comparing them with the original model using evaluation metrics. The comparison results are as follows: Figure 8 and Figure 9 As shown, from Figure 8 As can be seen, this invention improves model inference speed by 6.1%, accuracy by 13.2%, and average precision by 4.9%. Figure 9 As can be seen, the improved model of this invention has good detection performance.

[0165] It should be noted that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

[0166] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for detecting surface defects in steel that integrates state space and attention, characterized in that, include: Step S1: Obtain the steel surface defect dataset and preprocess the dataset to obtain dataset images for training; Step S2: Using YOLOv8 as the base network, the improved YOLOv8 network backbone network structure replaces the Bottleneck in C2f by introducing an improved C2f_LS module; replaces the SPPF module at the end of the original backbone with a multi-expansion channel refinement module; and embeds a cross-axis fusion modulation attention module at the backbone output to jointly refine the channel and spatial features. Step S3: Construct a state-space-based bidirectional feature pyramid neck network, use the weighted topology of BiFPN to construct a bidirectional fusion path, and introduce a lightweight visual state space module to replace the original feature extraction module at the feature fusion node. Use the state space model to perform dynamic filtering and long-distance dependency modeling on multi-scale features. Step S4: Input the training dataset images obtained in step S1 into the improved YOLOv8 network constructed by steps S2 and S3, set the initial training parameters, and obtain a deep learning model for steel surface defect detection through training. Step S5: Input the image of the steel surface to be detected into the model obtained in step S4 for detection, and obtain the target detection result containing the defect category and bounding box coordinates.

2. The method for detecting surface defects in steel by fusing state space and attention as described in claim 1, characterized in that: The specific implementation method of step S1 includes the following steps: Step S11: Obtain the NEU-DET dataset for steel surface defect detection, convert the obtained images and annotation files to a new format, and finally obtain the dataset images in YOLO format. Step S12: Divide the YOLO format dataset images into training set, validation set and test set according to a preset ratio of 8:1:1; Step S13: Apply the Mosaic data augmentation method to the training set to obtain the augmented training set images, and adjust the resolution of the augmented images to 512×512.

3. The method for detecting surface defects in steel by fusing state space and attention as described in claim 1, characterized in that, The improved YOLOv8 backbone network structure in step S2 is constructed using the following method: Step S21: Construct a C2f_LS module to replace the C2f module in the original backbone network; the C2f_LS module is constructed through a dual-stream decoupling mechanism, wherein the local stream branch uses RepVGGDW and ECAA attention concatenation to extract texture details, and the global stream branch uses large kernel convolution combined with large kernel perception and small kernel aggregation strategy to extract wide-area background information. Step S22: Replace the SPPF module at the end of the original backbone with a multi-channel dilatation refinement module; the MDCR module is constructed by parallel depth-separable dilatation convolution, each branch is set with a different dilatation rate to cover multiple levels of receptive fields, and finally lossless context aggregation is achieved by channel splicing. Step S23: Introduce a cross-axis fusion modulation attention module at the output of a specific feature layer of the backbone network; This module generates attention weights through channel gating and spatial gating, respectively, and performs cross-axis multiplication modulation to enhance the discriminative power of the features.

4. The method for detecting surface defects in steel by fusing state space and attention as described in claim 3, characterized in that, The method for constructing the C2f_LS module in step S21 includes the following steps: The C2f_LS module adopts a dual-stream parallel structure design, where the input feature X within the module is simultaneously fed into the local perception branch and the global perception branch for processing. Branch A, the local perception branch, extracts features using reparameterized convolution combined with an ECA attention module; its output... The calculation is as follows: in, For reparameterized convolution output, For channel attention weights, For global average pooling, It is the Sigmoid activation function. The feature map tensor input to this branch, For reparameterized Convolution kernel weight matrix, This is the reparameterized bias vector. For a one-dimensional convolution operation with a kernel size of k, For the generated channel attention weight vector, For the forward feedback neural network transformation, This is to output the enhanced feature map of the final output of the local perceptual branch; Branch B, the global perception branch, utilizes the LSConv module to extract wide-area background information. LSConv consists of a large kernel perception module and a small kernel aggregation module. First, LKP is used to generate dynamically selected weights. Then, adaptive aggregation is performed using SKA, the process of which is expressed as follows: Where X is the feature map input to this branch. for Convolution operation, The kernel size is Depth convolution operation, For normalization operations, Dynamically selectable weight tensors for large kernel perception. This is a small-core aggregation operation. For batch normalization operations, To output a feature map with wide-area background information from the global perception branch. For the forward feedback neural network transformation, The final output of the global perception branch is the enhanced feature map. Feature fusion: The outputs of branch A and branch B are concatenated and merged to obtain the final output. : Where X is the feature map input to this branch. for Convolution operation, For channel fusion operation, To output the enhanced feature map of the final output of the local perceptual branch, The final output of the global perception branch is the enhanced feature map. The outputs of branch A and branch B are spliced ​​and merged.

5. The method for detecting surface defects in steel by fusing state space and attention as described in claim 3, characterized in that, The method for constructing the MDCR module in step S22 includes the following steps: Feature segmentation: The input feature tensor is uniformly divided into 4 sub-tensors along the channel dimension. ; Multi-scale hole detection: Four sub-tensors are fed into parallel depthwise separable convolution branches, each with a set dilation rate. The output of the i-th branch The calculation formula is: in Let i be the i-th sub-feature tensor after segmentation. Let be the dilation rate of the dilated convolution corresponding to the i-th branch. For the expansion rate Depth-separable convolution, The output feature of the i-th branch; Cross-channel interaction and aggregation: to Channel-by-channel hybrid encoding and fusion are performed to obtain the module output. : in This represents the feature of the j-th channel output from the i-th branch. and For 1×1 convolution, This is for channel splicing operations. The features after fusion of the j-th channel group, This is the multi-scale analysis text aggregation feature map that is the final output of the MDCR module.

6. The method for detecting surface defects in steel by fusing state space and attention as described in claim 3, characterized in that, The method for constructing the CAFMAttention module in step S23 includes the following steps: Local convolutional branch: Fully connected (FC) layers are used to aggregate multi-head features, and local modulation features are obtained through grouped depthwise convolution. : Where Q, K, and V are the query, key, and value tensors obtained from the input feature mapping, respectively. For dimensional reshaping operations, For fully connected layer operations, For grouped depthwise convolution operations, Features extracted from local convolutional branches; Global self-attention branch: Introducing a learnable temperature parameter Calculate the cross-axis attention map to obtain global modulation features. : Where Q, K, and V are the query, key, and value tensors obtained from the input feature mapping, respectively. This is the transpose of the key tensor. For learnable temperature parameters, For normalized exponential functions, The calculated global attention map, For the convolution weights of the output projection layer, Features extracted for the global attention branch; Cross-axis feature fusion: Summing local and global features for output. in Features extracted for the global attention branch Features extracted from local convolutional branches This is the feature map output by the cross-axis fusion modulation attention mechanism module.

7. The method for detecting surface defects in steel by fusing state space and attention as described in claim 1, characterized in that, The state-space bidirectional feature pyramid neck network in step S3 is constructed using the following method: Step S31: Construct a bidirectional weighted feature pyramid structure based on the BiFPN topology, including a top-down upsampling fusion path and a bottom-up downsampling fusion path, and use weighted addition to fuse feature flows at different scales; Step S32: Construct a lightweight visual state space module to replace the feature extraction module in the neck network; The VSSLite module employs a three-branch parallel structure: the first branch is an SS2D unit based on a state-space model, used to establish global long-range dependencies; the second branch is a 3×3 depthwise separable convolution, used to extract local features; and the third branch is a shortcut connection. Step S33, the specific implementation of the SS2D unit is as follows: the input feature map is unfolded into a one-dimensional sequence along four directions, the sequence is selectively scanned and the state is updated using the S6 block, and finally it is restored into a two-dimensional feature map through cross-merging; Step S34: Aggregate the output features of the three branches, recalibrate the features through the SE channel attention module, and finally output a high-quality fused feature map through residual connection.

8. The method for detecting surface defects in steel by fusing state space and attention as described in claim 7, characterized in that, The weighted summation method in step S31 is expressed by the following formula: in For input features, For learnable weights, To prevent the minimum value of division by zero, This is the output feature map after fusion.

9. The method for detecting surface defects in steel by fusing state space and attention as described in claim 7, characterized in that, The discretized state-space model formula for the SS2D unit in step S33 is as follows: in, Given the input sequence, , , For time scale parameters, B is the state transition matrix, and B and C are the projection matrices. These are the state transition parameters after discretization. These are the discretized input parameters. Let be the hidden state variable of the previous time step t-1. Let be the hidden state variable at time t, where t is the time step index of the sequence. Output the sequence elements at time t; The formula for constructing the local convolutional branch of the VSSLite module in step S32 is as follows: in The feature tensor after input projection. The kernel size is Depth-separable convolution operations, For batch normalization operations, Let Gaussian error be the activation function of the linear unit. for Pointwise convolution operation, These are the output features of the local convolutional branch.

10. The method for detecting surface defects in steel by fusing state space and attention as described in claim 1, characterized in that, The network training in step S4 includes the following steps: Set the model training parameters: Use the SGD optimizer, set the momentum to 0.937, and use CIoU to calculate the bounding box regression loss, use binary cross-entropy loss to calculate the target confidence loss, and use multi-class cross-entropy loss to calculate the classification loss; update the weight parameters through backpropagation to obtain the deep learning model.

Images