Steel surface defect detection method and device

Abstract

The invention provides a steel surface defect detection method and device, and belongs to the technical field of steel defect detection.The steel surface defect detection method comprises the steps that a defect detection network based on a YOLO architecture is constructed, and the defect detection network comprises a lightweight feature extraction network, a self-adaptive multi-scale feature fusion network and a head network; the lightweight feature extraction network comprises four layers of architectures, and each layer of architecture comprises a first convolutional layer and a space expansion feature module; the feature space expansion module comprises a second convolutional layer and an element multiplication operator module; the adaptive multi-scale feature fusion network comprises an adaptive fusion module and a feature space expansion module; training the constructed defect detection network based on the historical steel defect image, and performing channel pruning on the trained defect detection network by using LAMP (Loop-Mediated Isothermal Amplification); and acquiring a to-be-identified steel image, and inputting the to-be-identified steel image into the pruned defect detection network for defect detection. According to the method, the defect detection precision of the network can be improved while the network complexity is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of steel defect detection, and in particular relates to a steel surface defect detection method and device. Background Art

[0002] Steel is one of the country's important industrial raw materials and is widely used in industrial fields such as machinery manufacturing, chemical equipment, and aerospace. However, due to a series of uncontrollable factors such as production technology, production environment, and raw materials, surface defects such as scratches and patches often appear on the surface of steel during the production process, which seriously affects the production quality of steel. Therefore, accurate and rapid detection of defects on the surface of steel is an important link in improving the production quality and production efficiency of steel.

[0003] In recent years, with the continuous development of deep learning related technologies, image processing methods based on deep learning have been widely studied. Among them, the classic two-stage detection network represented by Faster R-CNN and the end-to-end single-stage detection network represented by YOLO are widely used in the field of defect detection.

[0004] In the prior art, a variety of improved networks have been proposed, which can be used for defect detection on the surface of steel. For example, through a DCAN network based on deformable convolution and attention mechanism, the data enhancement method is first used to improve the contrast of steel sample data; secondly, the positioning ability of the network is effectively improved by proposing an enhanced deformation feature extraction module, which effectively improves the defect detection accuracy of the network. Through the ECC-YOLO network, the feature enhancement module ConvNeXt is introduced to improve the information representation ability of the feature extraction network. At the same time, the C2fFB module is designed to enhance the extraction ability of target feature information. Finally, the MPCE module is designed to reduce the interference of complex background information, which effectively improves the defect detection efficiency of the network. Through the improved YOLOv5s network, the CBAM attention mechanism is used to optimize the C3 module in the YOLOv5s network, and then CARAFE is used as an upsampling module. At the same time, a pyramid structure that integrates cross-stage local networks is proposed, thereby improving the defect detection accuracy of the network. Through a YOLO-Steel steel surface defect detection network based on multi-degree lightweight attention, by combining a lightweight channel attention module and a pyramid attention structure, the network's attention to effective features is enhanced, thereby improving the network's defect detection accuracy.

[0005] Existing technologies have optimized and improved the single-stage target detection network from different perspectives, effectively improving the network's detection accuracy in steel surface defect detection tasks. However, the complexity of the network has also increased, and in actual applications, the resource limitations of terminal embedded devices make it difficult to deploy, thus affecting the implementation of steel surface defect detection tasks. Summary of the invention

[0006] In order to solve the problem in the prior art that resource limitations of terminal embedded devices make it difficult to deploy complex networks with high precision, thus affecting the implementation of steel surface defect detection tasks, the present invention provides a steel surface defect detection method and device.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A method for detecting surface defects of steel, comprising:

[0009] A defect detection network based on the YOLO architecture is constructed, wherein the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network and a head network; the lightweight feature extraction network includes a four-layer architecture, each layer of which includes a first convolution layer and a space expansion feature module; the feature space expansion module includes a second convolution layer and an element multiplication operator module; the adaptive multi-scale feature fusion network includes an adaptive fusion module and a feature space expansion module;

[0010] The constructed defect detection network is trained based on historical steel defect images, and the trained defect detection network is pruned using LAMP to obtain the target defect detection network.

[0011] A steel image to be identified is obtained, and the steel image to be identified is input into the target defect detection network. The steel image to be identified is convolved by the first convolutional layer to obtain a feature map. The feature map is enhanced by the second convolutional layer, and the element multiplication operator module multiplies the features in the enhanced feature map to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features. The adaptive fusion module adaptively assigns attention weights to the feature map containing nonlinear features, and the feature space expansion module performs feature extraction to obtain a fused feature map. The fused feature map is decoded by the head network to determine the location and category of the steel surface defects.

[0012] Optionally, the first layer of the lightweight feature extraction network includes an edge feature extraction module, and the edge feature extraction module includes a Sobel operator branch and a maximum pooling branch. The Sobel operator branch extracts edge features of the steel image to be identified through the Sobel operator, and the maximum pooling branch extracts spatial information of defects through the maximum pooling operation.

[0013] Optionally, the adaptive fusion module includes a channel prior convolution attention module, and adaptively assigning attention weights to feature maps containing nonlinear features through the adaptive fusion module includes:

[0014] Two feature maps with the same number of channels are concatenated in the channel dimension, and the channel prior convolutional attention module is used to generate channel attention weights and spatial attention weights based on the concatenated feature maps;

[0015] The feature maps are weighted based on channel attention weights and spatial attention weights.

[0016] Optionally, the space expansion feature module includes two second convolutional layers and a concatenation layer, and feature extraction is performed through the feature space expansion module, including:

[0017] Performing a convolution operation on the input feature map through the first and second convolution layers to obtain a first feature map;

[0018] The element multiplication operator module performs a multiplication operation on the features in the first feature map to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features;

[0019] The feature maps of different levels are spliced ​​in the channel dimension through the splicing layer, and the spliced ​​feature maps are convolved again through the second convolutional layer to extract features.

[0020] Optionally, the head network is an anchor-free head network, and the fused features are decoded by the head network to determine the location and category of the steel surface defects, including:

[0021] The head network without anchor frames predicts each pixel of the input feature map to determine whether the pixel is the center point of the defect;

[0022] For the pixel at the center point, predict the width and height of the defect at the center point and determine the bounding box;

[0023] Predict the category label of the center point and identify the defect type based on the category label.

[0024] Optionally, the trained defect detection network is pruned using LAMP to obtain a target defect detection network, including:

[0025] Determine the magnitude of the weight as the pruning index, and calculate the LAMP score of each channel in the trained defect detection network; the larger the magnitude of the weight, the larger the LAMP score;

[0026] Pruning starts from the channel with the lowest LAMP score until the target sparsity is met and the target defect detection network is obtained.

[0027] Optionally, the element multiplication operator module is composed of two deep convolutional layers, two fully connected layers, a ReLu6 activation function and an element multiplication operator. When defect detection is performed on the steel image to be inspected, a deep separable convolution operation is performed through the first deep convolutional layer to extract features, and the extracted features are respectively transformed in feature dimension through two fully connected layers; a nonlinear transformation is performed on the features after the dimension transformation of one of the fully connected layers through the ReLu6 activation function; two feature maps that have undergone nonlinear transformation and one that have not undergone nonlinear transformation are multiplied element by element through the element multiplication operator to generate a new feature map, and a deep separable convolution operation is performed again on the new feature map through the second deep convolutional layer to extract features.

[0028] The present invention also provides a steel surface defect detection device, comprising:

[0029] A network construction module, used to construct a defect detection network based on the YOLO architecture, wherein the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network and a head network; the lightweight feature extraction network includes a four-layer architecture, each layer of which includes a first convolution layer and a space expansion feature module; the feature space expansion module includes a second convolution layer and an element multiplication operator module; the adaptive multi-scale feature fusion network includes an adaptive fusion module and a feature space expansion module;

[0030] The training module is used to train the constructed defect detection network based on historical steel defect images, and use LAMP to perform channel pruning on the trained defect detection network to obtain the target defect detection network;

[0031] The defect detection module is used to obtain a steel image to be identified, input the steel image to be identified into the target defect detection network, perform a convolution operation on the steel image to be identified through the first convolution layer to obtain a feature map; perform a feature enhancement operation on the feature map through the second convolution layer, and perform a multiplication operation on the features in the enhanced feature map by the element multiplication operator module to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features; adaptively assign attention weights to the feature map containing nonlinear features through the adaptive fusion module, and perform feature extraction by the feature space expansion module to obtain a fused feature map; decode the fused feature map through the head network to determine the location and category of the surface defects of the steel.

[0032] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned steel surface defect detection method when executing the program.

[0033] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned steel surface defect detection method is implemented.

[0034] The steel surface defect detection method provided by the present invention has the following beneficial effects:

[0035] In the defect detection network constructed by the present invention, both the lightweight feature extraction network and the adaptive multi-scale feature fusion network extract features through a space expansion feature module, and the space expansion feature module includes an element multiplication operator module. When defect detection is performed on the surface of steel, the element multiplication operator module performs a multiplication operation on the features in the feature map to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features. Since the element multiplication operator can generate a high-dimensional nonlinear feature space while performing low-dimensional space operations, this feature enables the defect detection network to achieve expansion and complexity of the feature space through the operation process itself without directly increasing the dimension of input data, and expand the implicit feature space of the feature enhancement module without increasing the complexity of the network, so as to achieve more effective extraction and fusion of steel surface defect features and achieve defect detection with higher accuracy. Secondly, the trained defect detection network uses LAMP to perform channel pruning, which can prune channels with lower contribution, so that the network can significantly reduce the complexity of the network while maintaining the original accuracy. In summary, the present invention combines element multiplication operators with channel pruning to reduce the complexity of the network while improving the defect detection accuracy of the network, so that the defect detection network can be deployed on terminal devices with limited resources and effectively carry out steel surface defect detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic flow chart of a steel surface defect detection method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the structure of a feature space expansion module provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the structure of an element multiplication operator module provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of the structure of a lightweight feature extraction network provided by an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of the structure of an AW-Fusion module provided in an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of the structure of an adaptive multi-scale feature fusion network provided by an embodiment of the present invention;

[0043] Figure 7 A schematic diagram of the structure of a head network based on no anchor frame provided by an embodiment of the present invention;

[0044] Figure 8 A schematic diagram of defect types provided by an embodiment of the present invention; wherein: Figure 8 Figure (a) shows a plaque. Figure 8 Figure (b) shows a scratch. Figure 8 Figure (c) shows a crack. Figure 8 Figure (d) shows the pitting surface. Figure 8 Figure (e) shows inclusions. Figure 8 Figure (f) shows oxidized rolled skin;

[0045] Fig. 9 A comparison diagram of the number of network channels before and after pruning provided by an embodiment of the present invention;

[0046] Fig.10 This is a comparison chart of the actual detection effects of different detection networks. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0048] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0049] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.

[0050] Figure 1 A schematic flow chart of a steel surface defect detection method provided in an embodiment of the present invention.

[0051] For example, Figure 1 As shown, the method comprises the following steps:

[0052] S1: Build a defect detection network based on the YOLO architecture.

[0053] In an embodiment of the present invention, the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network and a head network, and the lightweight feature extraction network and the adaptive multi-scale feature fusion network correspond to the backbone network and the neck network in the YOLO architecture respectively.

[0054] It should be understood that the feature extraction network is the main component of the defect detection network. By extracting feature information of different scales of the input image, it provides shallow detail feature information and deep semantic feature information for the feature fusion network. Most of the current feature extraction networks have the disadvantages of high complexity and weak extraction ability due to the limitations of their own design, and it is difficult to meet the needs of lightweight defect detection network design. Therefore, the present invention constructs a feature space expansion module, and then constructs a lightweight defect feature extraction network based on the feature space expansion module, aiming to effectively capture the complex defect characteristics of the steel surface.

[0055] Figure 2 A schematic diagram of the structure of a feature space expansion module provided in an embodiment of the present invention.

[0056] For example, Figure 2 As shown, the feature space expansion module mainly includes two second convolutional layers (the first second convolutional layer close to the input end and the second second convolutional layer close to the output end), a residual connection, an element multiplication operator module and a connection layer.

[0057] It should be understood that residual connections can effectively solve the problem of network gradient disappearance, so residual connections are introduced in the feature space expansion module so that the network can learn the direct mapping relationship between input and output. The two second convolutional layers enable the module to perform feature transformation on the input data, extract the feature information of the input data, and then store the extracted defect feature information in the channel dimension space.

[0058] The feature space expansion module is mainly used to extract features. The specific process is as follows: the first feature map is obtained by performing a convolution operation on the input feature map through the first second convolution layer; the feature in the first feature map is multiplied by the element multiplication operator module to map the feature to a high-dimensional nonlinear feature space to obtain nonlinear features; the feature maps of different levels are spliced ​​in the channel dimension through the splicing layer, and the spliced ​​feature map is convolved again through the second second convolution layer to extract features.

[0059] Figure 3 A schematic diagram of the structure of an element multiplication operator module provided in an embodiment of the present invention.

[0060] For example, Figure 3 As shown in the figure, the element multiplication operator module mainly includes two deep convolutional layers (the first deep convolutional layer close to the input end and the second deep convolutional layer close to the output end), a fully connected layer, a ReLu6 activation function, a residual connection and an element multiplication operator.

[0061] It should be understood that the deep convolution layer can effectively extract the multi-level features of the input data through multi-layer convolution operations, and the weight parameters of the convolution layer are shared at different positions, reducing the number of network parameters. After passing through the deep convolution layer, a branching structure is adopted, and through the fully connected layer, the fully connected layer can perform linear transformation and dimensionality reduction on high-dimensional features, so that the feature dimension meets the processing conditions of the element multiplication operator.

[0062] In addition, the ReLu6 activation function is introduced on one of the branches of the element multiplication operator module to introduce nonlinear features into the network. Compared with other activation functions, the output range of the ReLu6 activation function is limited, which can simplify the hardware implementation. For mobile devices and embedded systems, less bit width can be used for calculation, thereby reducing power consumption and resource requirements. The element multiplication operator is then used to multiply the feature map to expand the implicit feature space and improve the network's ability to extract and fuse different features.

[0063] When performing defect detection on the steel image to be inspected, the first deep convolutional layer performs a deep separable convolution operation to extract features, and the extracted features are transformed in feature dimension through two fully connected layers respectively; the features after dimension transformation of one of the fully connected layers are nonlinearly transformed through the ReLu6 activation function; the two feature maps that have undergone nonlinear transformation and those that have not undergone nonlinear transformation are multiplied element by element through the element multiplication operator to generate a new feature map, and the second deep convolutional layer performs a deep separable convolution operation on the new feature map to extract features.

[0064] The element multiplication operator is considered to be a method of operation that can map the input to a high-dimensional nonlinear feature space to improve the representation ability of the network. Specifically, in a single-layer neural network, the element multiplication operator is usually expressed as:

[0065] Y=(w1x+b1)*((w2x+b2) (1)

[0066] Among them, x and Y are the input and output of the neural network respectively, w1 and w2 are the different weights of the hidden layers in the neural network respectively, and b1 and b2 are the bias terms corresponding to their weights respectively.

[0067] For ease of computation, the element-wise multiplication operator can be simplified to:

[0068]

[0069] in,

[0070] If the number of input channels of the neural network is d, then and The dimension of is d+1, so the element-wise multiplication operator can be expressed as:

[0071]

[0072] Expand the above formula to get:

[0073]

[0074] in, i and j represent the channel index values. α represents the coefficient of each item, and its expression is:

[0075]

[0076] From the above expansion, we can see that in d-dimensional space, using the element multiplication operator, we can get: dimensional implicit feature space, thus significantly increasing the spatial dimension of the features without incurring additional computational complexity.

[0077] It can be seen that the element multiplication operator has a good effect in expanding the implicit feature space. Therefore, building an element multiplication operator module based on the element multiplication operator can improve the network's ability in feature extraction and fusion, and then enrich the feature expression ability by fusing features of different scales in the channel dimension.

[0078] Furthermore, a lightweight feature extraction network is constructed according to the above-mentioned space expansion feature module, and the lightweight feature extraction network here is a lightweight and efficient feature extraction network (Lightweight and Efficient Defect Feature Extraction Network, LENet).

[0079] Figure 4 A schematic diagram of the structure of a lightweight feature extraction network provided by an embodiment of the present invention.

[0080] For example, Figure 4 As shown, the lightweight feature extraction network adopts a four-layer hierarchical architecture. Each layer of the architecture includes a first convolutional layer and a space expansion feature module. When the image is input, a convolution operation is performed through the first convolutional layer to obtain a feature map, and then the feature space expansion module is used to extract features. The process of feature extraction by the feature space expansion module can refer to the description in the above embodiment.

[0081] Optionally, in order to adaptively fuse feature information of different scales and enhance the feature extraction capability of the network, a fast spatial pyramid pooling (SPPF) module is used at the end of the lightweight feature extraction network, such as Figure 4 As shown in Figure 1, the SPPF module includes a series of convolutional layers and maximum pooling layers, which realizes the fusion of features of different scales through convolution and pooling.

[0082] It should be understood that in steel surface defect detection, edge feature information can effectively help the network understand the category of defects. Therefore, in order to further improve the network's ability to extract edge feature information, the present invention uses an edge feature extraction module designed based on the Soble operator in the first layer of the network to perform edge feature extraction.

[0083] Depend on Figure 4 As shown in the figure, the edge feature extraction module includes a Sobel operator branch and a maximum pooling branch. The Sobel operator branch extracts the edge features of the steel image to be identified through the Sobel operator. The Sobel operator senses the horizontal and vertical edge features of the image through two 3D convolutions. At the same time, in the maximum pooling branch, the maximum pooling operation is used to extract the spatial information of the defect. Finally, the spatial features and edge features are simultaneously input into the next layer of the network for further feature extraction.

[0084] In the defect detection network, the feature fusion network is the core component, and its purpose is to enable the network to understand the information of the input image more comprehensively by further fusing feature information from different scales of the backbone network.

[0085] In an embodiment of the present invention, the feature fusion network is an adaptive multi-scale feature fusion network (Adaptive Multi-scale Feature Fusion Network, AMFN), which includes an adaptive fusion module (AW-Fusion) and a feature space expansion module.

[0086] It should be understood that in a feature fusion network, the fusion mechanism of the network is crucial to the performance of the network. In order to improve the performance of the feature fusion network and reduce the complexity of the network, the present invention constructs an adaptive multi-scale feature fusion network based on an adaptive fusion module and a feature space expansion module.

[0087] Among them, the AW-Fusion module is mainly a fusion mechanism with adaptive weight allocation designed for two input features with the same number of channels.

[0088] Figure 5 A schematic diagram of the structure of an AW-Fusion module provided in an embodiment of the present invention.

[0089] For example, Figure 5 As shown in the figure, in the AW-Fusion module, the two input feature maps are first concatenated in the channel dimension, and then the attention weights are adaptively allocated in the channel and space dimensions through the Channel Prior Convolutional Attention (CPCA) module. The process is as follows:

[0090] Two feature maps with the same number of channels are spliced ​​in the channel dimension, and the channel attention weight and spatial attention weight are generated based on the spliced ​​feature maps through the channel prior convolutional attention module; the feature maps are weighted based on the channel attention weight and spatial attention weight.

[0091] Specifically, the CPCA module generates one-dimensional weights based on the input features through the channel attention mechanism, and these weights are multiplied element-by-element with the input features to effectively capture the key information of the channel dimension. Subsequently, the CPCA module generates three-dimensional spatial attention weights in the spatial dimension to enhance the learning of spatial saliency information. These spatial weights are multiplied with the features processed by the channel attention to output a more refined feature expression.

[0092] Optionally, the weighted feature map is added element-by-element to the original input feature map, and the results are integrated again in the channel dimension. This design not only enhances the representation ability of each feature map, but also further improves the complementary ability of feature information through the mutual fusion of features, thereby enhancing the feature fusion network's more comprehensive understanding of feature information.

[0093] Then, a feature-adaptive multi-scale feature fusion network is constructed based on the above-mentioned AW-Fusion module and feature space expansion module.

[0094] Figure 6 A schematic diagram of the structure of an adaptive multi-scale feature fusion network provided in an embodiment of the present invention.

[0095] For example, Figure 6 As shown in Figure 1, the adaptive multi-scale feature fusion network adopts a top-down, bottom-up and multi-scale fusion connection method, integrating downsampling, AW-Fusion module, feature space expansion module and convolution layer. The lightweight feature extraction network inputs features of different scales into the adaptive multi-scale feature fusion network for further fusion.

[0096] In an embodiment of the present invention, the feature extraction module of the adaptive multi-scale feature fusion network adopts a feature space expansion module, which can not only expand the implicit feature space through the element multiplication operator to further enhance the mining capability of the feature fusion network for redundant feature information, but also effectively reduce the complexity of the network. The adaptive network multi-scale feature fusion network adopts a top-down, bottom-up and multi-scale fusion connection mode. It can achieve effective utilization of features of different scales. In addition, the network designs an AW-Fusion module by combining the CPCA module. The AW-Fusion module automatically adjusts the fusion weight according to the importance of different feature layers, optimizes the information flow in the feature transmission process, and ensures the robustness and accuracy of the network when processing diversified inputs.

[0097] In the defect detection network, the head network decodes the feature information output from the feature fusion network into the location information and category information of the defect target through the detection head and the classification head.

[0098] In the present invention, a head network without anchor boxes is used to predict whether each pixel of the feature map is the center point of the defect, thereby realizing defect location. Secondly, for the pixel identified as the center point, the width and height of the center point defect are further predicted to determine the bounding box. At the same time, the center point will also have a category label to identify the category of the defect.

[0099] Figure 7 A schematic diagram of the structure of a head network based on no anchor frame provided in an embodiment of the present invention.

[0100] For example, Figure 7 As shown in the figure, the detection head decodes the feature information by using a convolutional layer to generate detection results, including the coordinates of the bounding box. In addition, the classification head classifies the feature map through a convolutional layer and outputs the category and the confidence of the category.

[0101] In actual application, the role of the head network is to decode the fused features and determine the location and category of the surface defects of the steel. The specific implementation process is: predict each pixel of the input feature map through the head network without anchor boxes to determine whether the pixel is the center point of the defect; for the pixel of the center point, predict the width and height of the center point defect and determine the bounding box; predict the category label of the center point and identify the defect type based on the category label. Among them, predicting the category label of the center point can be achieved by applying a classifier to each pixel of the feature map, and outputting the category probability of the pixel corresponding to the target. Finally, the category with the highest probability is selected as the category label of the center point.

[0102] S2: The constructed defect detection network is trained based on historical steel defect images, and the trained defect detection network is pruned using LAMP to obtain the target defect detection network.

[0103] The historical steel defect images selected by the present invention cover the six most common surface defects in the steel production process, including Figure 8 As shown, Figure 8 Figure (a) shows a plaque. Figure 8 Figure (b) shows a scratch. Figure 8 Figure (c) shows a crack. Figure 8 Figure (d) shows the pitting surface. Figure 8 Figure (e) shows inclusions. Figure 8 Figure (f) shows oxidized rolled skin.

[0104] Among them, there are 300 defect images of each type, with the category and location of the defects marked in detail. The historical steel defect images are divided into training set, test set and verification set in a ratio of 7:2:1.

[0105] The constructed defect detection network is trained through the training set; the validation set is used to evaluate the performance of the model during the training process and help select the best hyperparameters; the test set is used to test the test set and finally evaluate the performance of the model after the model training is completed.

[0106] LAMP (Layer-wise Adaptive Magnitude Pruning) is a channel pruning method that prunes based on the magnitude of weights and adaptively determines the sparsity between layers. In a neural network, each fully connected layer or convolutional layer has a weight, and the magnitude is the absolute value of the weight. The LAMP score is the basis for evaluating the contribution of each channel to the network during the pruning process.

[0107] The process of using LAMP to perform channel pruning on the defect detection network is as follows: Determine the magnitude of the weights as the pruning index, and calculate the LAMP score of each channel in the trained defect detection network; start pruning from the channel with the lowest LAMP score until the target sparsity is met, obtaining the target defect detection network.

[0108] In the embodiments of the present invention, the design principle of the LAMP score is as follows:

[0109] For a feedforward neural network with a depth of d, the weight tensor of each layer is W (1) , …, W d . To make the designed LAMP score unified. Since the weight tensor dimensions of the fully connected layer and the convolutional layer are different, each weight tensor is flattened into a one-dimensional vector, that is, W[1], …, W[d]. Assume that the magnitudes of the weights are sorted according to the index value size, then when μ < v, then W[μ] < W[v]. Then the LAMP score of the weight W[u] is:

[0110]

[0111] where u and v represent the index values of the weights, and W[u] and W[u] represent the weight terms mapped by u and v.

[0112] It can be seen from the above formula (6) that the LAMP score measures the importance of the target connection relative to other connections in the same layer. Since represents the square of the target connection weight, and represents the sum of the squares of the weights of all connections in the same layer that are more important than the target connection, therefore:

[0113]

[0114] That is, the larger the magnitude value, the larger the LAMP score value, and then prune the connections with low LAMP scores until the required sparsity is met.

[0115] In each layer, to avoid layer collapse, the LAMP score has at least one optimal connection with a value of 1 in each layer. Therefore, after determining the sparsity rate, select the smallest LAMP score for pruning. The designs of formulas (6) and (7) enable pruning to be adaptively based on the magnitude until the sparsity rate is met.

[0116] In the above embodiment, after the training of the defect detection network is completed, the network is compressed at the back end through channel pruning, which can further reduce the complexity of the defect detection network and improve the detection speed of the network.

[0117] S3: Obtain a steel image to be identified, input the steel image to be identified into a target defect detection network, and perform defect detection.

[0118] Based on the target defect detection network constructed as above, the steel image to be identified is input into the target defect detection network, and the steel image to be identified is convolved by the first convolution layer to obtain a feature map; the feature map is enhanced by the second convolution layer, and the element multiplication operator module multiplies the features in the enhanced feature map to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features; the feature map containing nonlinear features is adaptively assigned attention weights by the adaptive fusion module, and the feature space expansion module performs feature extraction to obtain a fused feature map; the fused feature map is decoded by the head network to determine the location and category of the steel surface defects.

[0119] In the defect detection network constructed by the present invention, both the lightweight feature extraction network and the adaptive multi-scale feature fusion network extract features through the space expansion feature module, and the space expansion feature module includes an element multiplication operator module. Since the element multiplication operator can generate a high-dimensional nonlinear feature space while performing low-dimensional space operations, this feature enables the defect detection network to achieve the expansion and complexity of the feature space through the operation process itself without directly increasing the dimension of the input data, and expand the implicit feature space of the feature enhancement module without increasing the complexity of the network, so as to achieve more effective extraction and fusion of the surface defect features of steel, and achieve higher-precision defect detection; secondly, the trained defect detection network uses LAMP for channel pruning, which can prune channels with lower contribution, so that the network can significantly reduce the complexity of the network while maintaining the original accuracy. In summary, the present invention combines element multiplication operators and channel pruning, which can improve the detection accuracy of the defect detection network while reducing the complexity of the network, so that the defect detection network can be deployed on terminal devices with limited resources, and effectively carry out the task of steel surface defect detection.

[0120] Next, the effect of the defect detection network constructed by the present invention is verified through experiments and analysis.

[0121] First, select the dataset.

[0122] The data set selected by the present invention covers the six most common surface defects in the steel production process, namely, plaques, scratches, crack pitting, inclusions and oxide scale.

[0123] Among them, there are 300 defect images of each type, with the category and location of the defects marked in detail. The data set is divided into training set, test set and validation set in a ratio of 7:2:1.

[0124] Secondly, arrange the experimental environment and set the network hyperparameters.

[0125] The specific experimental environment is: windows1l, 64-bit operating system, GPU uses NVIDIA GeForce RTX3060ti, CPU uses 13th Gen Intel(R)Core(TM)i5-13490F, deep learning framework uses PyTorch+CUDA11.6, and programming language uses Python3.8.

[0126] The hyperparameter settings of the network are shown in Table 1.

[0127] Table 1 Network hyperparameter settings

[0128] Hyperparameters set up Training batches 32 Iterations 300 Initial learning rate 0.001 Image size 640 momentum 0.937 Optimizer SGD

[0129] In order to more fully verify the effectiveness and superiority of the improved network of the present invention, quantitative analysis is performed from three aspects: network complexity, network detection accuracy and network detection speed. The relevant evaluation index formula is as follows:

[0130]

[0131] Among them, P is the network accuracy, R is the recall rate of the network, AP is the average precision, and mAP is the mean average precision. These indicators are usually used to measure the detection accuracy of the network.

[0132] Params=C out ×(K h ×K w ×C in +1) (12)

[0133] FLOPs = 2 × C in ×C out ×K h ×K w ×H out ×W out (13)

[0134] Among them, Params represents the number of network parameters, FLOPs represents the floating-point operations of the network, and C in and C out Respectively represent the number of input and output channels, H out and W out are the height and width of the output feature map, respectively, and K h and Kw are the height and width of the convolution kernel, which are usually used to measure the complexity of the network.

[0135]

[0136] Among them, FPS represents the frame rate of network detection, ToP represents the total number of images, TOT represents the total detection time, and FPS is usually used to measure the detection speed of the network.

[0137] Then, comparative experiments are carried out. The comparative experiments are divided into comparative experiments of different feature extraction networks, comparative experiments of different feature fusion networks, comparative experiments of different attention mechanisms in AMFN, comparative experiments of different hyperparameters of channel pruning, comparative experiments of different detection networks and qualitative analysis of the network. Below, each comparative experiment is explained separately.

[0138] Comparative experiments of different feature extraction networks:

[0139] In order to verify the superiority of the lightweight and efficient feature extraction network (LENet) provided by the present invention over other networks in the steel surface defect detection task. The embodiment of the present invention selects a series of efficient feature extraction networks for comparative experiments, including: FasterNet, EffcientVit, GhostNet, MobileNetV4, MnasNet and TinyNe, and the feature fusion network adopts the AMFN in the present invention.

[0140] The comparative experimental results are shown in Table 2.

[0141] Table 2 Comparative experimental results of different feature extraction networks

[0142]

[0143] As shown in Table 2, in terms of network complexity, considering the network parameters and floating-point operations, the parameters of GhostNet, MnasNet, and TinyNet are all lower than LENet, but the floating-point operations of LENet are significantly lower than those of the above networks, which is only 3.1×10 9 Therefore, overall, LENet is superior to the other networks mentioned above in terms of network complexity.

[0144] Secondly, in terms of the detection accuracy of the network, LENet's mAP is 79.4%, which is only slightly lower than FasterNet and MobileNetv4, but compared with GhostNet, MnasNet and TinyNet, LENet is superior to them in detection accuracy.

[0145] Finally, in terms of network detection speed, LENet’s FPS reaches 176.9 frames / s, which is better than other feature extraction networks.

[0146] Therefore, considering the three aspects of complexity, detection accuracy and speed, LENet achieves high detection accuracy and detection speed under the condition of low network complexity. The experimental results verify that the element multiplication operator generates a high-dimensional feature space while operating in a low-dimensional space, effectively improving the network's ability to extract features, making the network more suitable for deployment in embedded devices with scarce computing resources to carry out steel surface defect detection tasks.

[0147] Comparative experiments of different feature fusion networks:

[0148] In order to verify the superiority of the adaptive multi-scale fusion network provided by the present invention in processing the feature information of different scales of the feature extraction network. The embodiment of the present invention selects a series of feature fusion networks that show excellent effects in the field of target detection for comparative experiments, including: Feature pyramid networks (FPN), path aggregation network (PAN), bidirectional feature pyramid network (BiFPN), asymptotic feature pyramid network (AFPN), global feature pyramid network (GFPN) and AMFN. The feature extraction networks all use LENet. The results of the comparative experiments are shown in Table 3.

[0149] Table 3 Comparative experimental results of different feature fusion networks

[0150] Feature extraction network Feature fusion network <![CDATA[Params / 10 6 ]]> <![CDATA[FLOPs / 10 9 ]]> mAP / % FPS / (frame / s) LENet FPN 2.68 2.5 77.9 164.5 LENet PAN 2.71 2.8 76.8 172.3 LENet BiFPN 1.71 2.2 77.4 175.5 LENet AFPN 2.29 2.5 77.6 166.8 LENet GFPN 2.92 2.5 76.0 163.9 LENet AMFN 3.29 3.1 79.4 176.9

[0151] It can be seen from Table 3 that, firstly, in terms of network complexity, since the feature extraction network used in this comparative experiment is LENet, all networks have lower complexity in terms of network parameter quantity and floating-point operation quantity.

[0152] However, in terms of network detection accuracy and detection speed, AMFN is significantly better than other feature fusion networks. This is due to the design of an adaptive fusion module in AMFN, which enables the network to perform weighted integration of multi-scale feature information for feature extraction from two dimensions: channel and space. For BiFPN, its network complexity is the lowest, but its detection accuracy is two percentage points lower than AMFN. And the detection speed is also lower than AMFN. Considering the three aspects of network complexity, detection accuracy and detection speed. The AMFN designed by the present invention has more advantages.

[0153] Comparative experiments of different attention mechanisms in AMFN:

[0154] In AMFN, the present invention designs an adaptive fusion module based on the attention mechanism, which fuses features of different scales with the same number of channels, and then uses the attention mechanism to further enhance the effective feature information. In order to verify the superiority of the CPCA attention selected in the present invention in processing the surface defect characteristics of steel. The experiment was conducted by setting up a control group without an attention mechanism and a comparison group of attention mechanisms including Squeeze-and-Excitation Blocks (SE), Simultaneous Attention to Multi-dimensions (SimAM), Multi-Level Channel Attention (MLCA), Multi-Path Channel Attention (MPCA) and Efficient Channel Attention (ELA). The results of the comparative experiment are shown in Table 4.

[0155] Table 4 Comparative experimental results of different attention mechanisms in AMFN

[0156]

[0157] As shown in Table 4, by comparing the group without attention mechanism and the group with attention mechanism, in terms of network complexity, the introduction of attention mechanism will increase the number of network parameters and floating point operations to a certain extent, and the introduction of attention that is not suitable for the network may lead to a decrease in the detection accuracy of the network. However, for CPCA attention, this attention dynamically weights the features from the two dimensions of channel and space. From the experimental data, the detection of three types of defects, plaque, scratch and oxidation rolling skin, has been significantly improved. The mAP of the network has increased by 1.3 percentage points compared to that without attention mechanism. Therefore, on the whole, it is effective to use CPCA attention to construct an adaptive fusion module in AMFN, and it is more advantageous than other attention mechanisms.

[0158] Comparative experiments of different hyperparameters for channel pruning:

[0159] In the study of lightweight networks, the lightweight front-end design of the network and the back-end compression of the network are crucial for the actual application of the network. In order to further reduce the complexity of the network and improve the detection speed of the network, the present invention adopts channel pruning to further compress the network. However, in pruning, the setting of relevant hyperparameters directly affects the actual performance of the network. The present invention conducts comparative experiments on the two hyperparameters of sparse rate and whether to prune globally, and the experimental results are shown in Table 5.

[0160] Table 5 Comparison experimental results of different pruning hyperparameters

[0161]

[0162] As shown in Table 5, using channel pruning to perform backend compression on the network effectively reduces the complexity of the network and improves the detection speed of the network. The number of network parameters and floating-point operations are reduced by more than 50%, which is friendly to low-memory terminal embedded devices.

[0163] At the same time, the detection speed of the network has also been improved to a certain extent. Under the same sparsity rate, global pruning will reduce the network complexity more than ordinary pruning, but the network detection accuracy will also be reduced more. In addition, when the sparsity rate is 4.0, the detection accuracy of the network drops significantly or even collapses. Therefore, considering the three aspects of network complexity, detection accuracy and detection speed, when the sparsity rate is 3.0 and global pruning is not used, compared with the original network, the number of parameters is reduced by 66.9%, the floating-point operations are reduced by 64.5%, the mAP is reduced by 1.3%, and the FPS is increased by 2.9 frames / s. It has better effects than other hyperparameter settings. The comparison of the number of channels in each layer of the network before and after the network adopts this set of hyperparameters is shown in the figure below. Fig. 9 shown.

[0164] Fig. 9 A comparison diagram of the number of network channels before and after pruning provided by an embodiment of the present invention, through Fig. 9 It can be seen intuitively that for most layers, pruning effectively removes channels with lower contributions.

[0165] Comparative experiments of different detection networks:

[0166] In order to further verify the superiority of the network and pruned network provided by the present invention, the embodiment of the present invention selects classic target detection networks such as Faster Region Convolutional Neural Network (Faster R-CNN) and Single Shot MultiBox Detector (SSD) to compare with the YOLO series network. The test results are shown in Table 6.

[0167] Table 6 Comparative test results of different networks

[0168]

[0169] As can be seen from Table 6, first of all, in terms of network complexity, the parameters and floating-point operations of the classic target detection network are too large to be applied in actual terminal devices. With the development of related technologies, the network complexity of the YOLO series detection network YOLOv8n has been effectively reduced, and the parameters and floating-point operations are only 2.87M and 8.1G. However, after pruning, the parameters and floating-point operations of the network proposed in the present invention are only 1.09M and 1.1G.

[0170] Secondly, in terms of detection accuracy, the mAP of the network proposed in the present invention is 79.4%, and 78.1% after pruning, which is also better than the above network.

[0171] Finally, in terms of detection speed, the FPS of the pruned network in the present invention reaches 179.8 frames / s, which is much higher than the other networks mentioned above.

[0172] It can be seen that, considering the network complexity, detection accuracy and detection speed, the network provided by the present invention effectively achieves a balance in complexity, detection accuracy and speed after channel pruning. Compared with other networks, this network has better advantages in the task of steel surface defect detection and is more suitable for the practical application of steel surface defect detection.

[0173] Qualitative analysis of the network:

[0174] In order to conduct a qualitative analysis of the comparative experimental results of different detection networks. The embodiment of the present invention compares the actual detection effect diagrams of the network of the present invention, the pruned network, and the YOLOv8n, YOLOv5s, and Faster R-CNN networks, such as Fig.10 shown.

[0175] exist Fig.10In the figure, different defects are marked with detection boxes of different colors, and the detection category and confidence are above the detection box. The network and pruned network proposed in the present invention can locate defects more accurately than other networks. Although the above networks can locate the position of defects more accurately, the confidence of the same defect recognition of the network proposed in the present invention and the pruned network is significantly higher than that of other networks.

[0176] In summary, the present invention aims at the problems of high complexity and low detection accuracy in steel surface defect detection networks based on deep learning. A steel surface defect detection network with channel pruning of a joint element multiplication operator is proposed. Through lightweight network design and back-end network compression technology, the network can detect surface defects in real time with high accuracy under low complexity. Through different comparative experiments, it is proved that the network proposed in the present invention is superior to other networks in the task of steel surface defect detection. The network parameter amount and floating-point operation amount after pruning of the present invention are only 1.09M and 1.1G, but the network's mAP can reach 78.1% and the detection speed can reach 194.8%. Therefore, the network constructed by the present invention achieves a good balance between network complexity and detection efficiency, and has good application prospects.

[0177] Based on the same inventive concept, an embodiment of the present invention further provides a steel surface defect detection device, which includes:

[0178] A network construction module is used to construct a defect detection network based on the YOLO architecture, wherein the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network and a head network; the lightweight feature extraction network includes a four-layer architecture, each layer of the architecture includes a first convolution layer and a space expansion feature module; the feature space expansion module includes a second convolution layer and an element multiplication operator module; the adaptive multi-scale feature fusion network includes an adaptive fusion module and a feature space expansion module.

[0179] The training module is used to train the constructed defect detection network based on historical steel defect images, and use LAMP to perform channel pruning on the trained defect detection network to obtain the target defect detection network.

[0180] The defect detection module is used to obtain a steel image to be identified, input the steel image to be identified into the target defect detection network, perform a convolution operation on the steel image to be identified through the first convolution layer to obtain a feature map; perform a feature enhancement operation on the feature map through the second convolution layer, and perform a multiplication operation on the features in the enhanced feature map by the element multiplication operator module to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features; adaptively assign attention weights to the feature map containing nonlinear features through the adaptive fusion module, and perform feature extraction by the feature space expansion module to obtain a fused feature map; decode the fused feature map through the head network to determine the location and category of the surface defects of the steel.

[0181] Each module in the above-mentioned steel surface defect detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0182] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the steel surface defect detection method embodiment. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0183] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the steel surface defect detection method can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0184] It will be appreciated by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0186] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0188] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. A method for detecting surface defects of steel, characterized in that: include: Constructing a defect detection network based on the YOLO architecture, the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network and a head network; the lightweight feature extraction network includes a four-layer architecture, each layer of which includes a first convolutional layer and a spatial expansion feature module; The feature space expansion module includes a second convolution layer and an element multiplication operator module; the adaptive multi-scale feature fusion network includes an adaptive fusion module and a feature space expansion module; The constructed defect detection network is trained based on historical steel defect images, and the trained defect detection network is pruned using LAMP to obtain the target defect detection network. Obtain a steel image to be identified, input the steel image to be identified into the target defect detection network, perform a convolution operation on the steel image to be identified through the first convolution layer to obtain a feature map; perform a feature enhancement operation on the feature map through the second convolution layer, and perform a multiplication operation on the features in the enhanced feature map through the element multiplication operator module to map the features to a high-dimensional nonlinear feature space to obtain a nonlinear feature; The adaptive fusion module adaptively assigns attention weights to the feature map containing nonlinear features, and the feature space expansion module extracts features to obtain the fused feature map; the fused feature map is decoded by the head network to determine the location and category of steel surface defects.

2. The method for detecting surface defects of steel according to claim 1, characterized in that: The first layer of the lightweight feature extraction network includes an edge feature extraction module, which includes a Sobel operator branch and a maximum pooling branch. The Sobel operator branch extracts edge features of the steel image to be identified through the Sobel operator, and the maximum pooling branch extracts spatial information of defects through the maximum pooling operation.

3. The method for detecting surface defects of steel according to claim 1 or 2, characterized in that: The adaptive fusion module includes a channel prior convolution attention module, and adaptively assigns attention weights to feature maps containing nonlinear features through the adaptive fusion module, including: Two feature maps with the same number of channels are concatenated in the channel dimension, and the channel prior convolutional attention module is used to generate channel attention weights and spatial attention weights based on the concatenated feature maps; The feature maps are weighted based on channel attention weights and spatial attention weights.

4. The method for detecting surface defects of steel according to claim 1 or 2, characterized in that: The spatial expansion feature module includes two second convolutional layers and a concatenation layer, and feature extraction is performed through the feature space expansion module, including: Performing a convolution operation on the input feature map through the first and second convolution layers to obtain a first feature map; The element multiplication operator module performs a multiplication operation on the features in the first feature map to map the features to a high-dimensional nonlinear feature space to obtain nonlinear features; The feature maps of different levels are spliced ​​in the channel dimension through the splicing layer, and the spliced ​​feature maps are convolved again through the second convolutional layer to extract features.

5. The method for detecting surface defects of steel according to claim 1 or 2, characterized in that: The head network is an anchor-free head network, which decodes the fused features to determine the location and category of the steel surface defects, including: The head network without anchor frames predicts each pixel of the input feature map to determine whether the pixel is the center point of the defect; For the pixel at the center point, predict the width and height of the defect at the center point and determine the bounding box; Predict the category label of the center point and identify the defect type based on the category label.

6. The method for detecting surface defects of steel according to claim 1 or 2, characterized in that: The trained defect detection network is pruned using LAMP to obtain the target defect detection network, including: Determine the magnitude of the weight as the pruning index, and calculate the LAMP score of each channel in the trained defect detection network; the larger the magnitude of the weight, the larger the LAMP score; Pruning starts from the channel with the lowest LAMP score until the target sparsity is met and the target defect detection network is obtained.

7. The method for detecting surface defects of steel according to claim 1 or 2, characterized in that: The element multiplication operator module is composed of two deep convolutional layers, two fully connected layers, a ReLu6 activation function and an element multiplication operator. When performing defect detection on the steel image to be inspected, a deep separable convolution operation is performed through the first deep convolutional layer to extract features, and the extracted features are respectively transformed in feature dimension through two fully connected layers; a nonlinear transformation is performed on the features after the dimension transformation of one of the fully connected layers through the ReLu6 activation function; two feature maps that have undergone nonlinear transformation and one that have not undergone nonlinear transformation are multiplied element by element through the element multiplication operator to generate a new feature map, and a deep separable convolution operation is performed again on the new feature map through the second deep convolutional layer to extract features.

8. A steel surface defect detection device, characterized in that: include: A network construction module, used to construct a defect detection network based on the YOLO architecture, wherein the defect detection network includes a lightweight feature extraction network, an adaptive multi-scale feature fusion network, and a head network; the lightweight feature extraction network includes a four-layer architecture, and each layer of the architecture includes a first convolution layer and a spatial expansion feature module; The feature space expansion module includes a second convolution layer and an element multiplication operator module; the adaptive multi-scale feature fusion network includes an adaptive fusion module and a feature space expansion module; The training module is used to train the constructed defect detection network based on historical steel defect images, and use LAMP to perform channel pruning on the trained defect detection network to obtain the target defect detection network; The defect detection module is used to obtain a steel image to be identified, input the steel image to be identified into the target defect detection network, perform a convolution operation on the steel image to be identified through the first convolution layer to obtain a feature map; perform a feature enhancement operation on the feature map through the second convolution layer, and perform a multiplication operation on the features in the enhanced feature map by the element multiplication operator module to map the features to a high-dimensional nonlinear feature space to obtain a nonlinear feature; The adaptive fusion module adaptively assigns attention weights to the feature map containing nonlinear features, and the feature space expansion module extracts features to obtain the fused feature map; the fused feature map is decoded by the head network to determine the location and category of steel surface defects.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting surface defects of steel materials according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting surface defects of steel materials according to any one of claims 1 to 7 is implemented.

Images