Training method of steel surface defect detection model

Abstract

The invention provides a training method of a steel surface defect detection model. The training method of the steel surface defect detection model comprises the following steps that S1, the steel surface defect detection model is initialized, the steel surface defect detection model is a YOLO v8 network, and a convolution-to-full connection C2f module in a backbone network of the YOLO v8 network adopts double branches for coding; s2, a training data set is constructed, the training data set comprises images of steel of different defect types and images of steel without defects, and each image further comprises the corresponding defect type as a label; and S3, training the steel surface defect detection model based on the training data set, and updating model parameters based on a preset loss function. Through a mode of double-branch parallel coding and final feature fusion, the feature extraction capability of a backbone network is improved, and the accuracy of a defect detection model is enhanced; and the calculation burden is reduced by introducing the dynamic convolution and the phantom convolution.

Description

Technical Field

[0001] The invention relates to the field of steel surface defect detection, and in particular to a training method for a steel surface defect detection model. Background Art

[0002] Steel is widely used in construction, bridges, aerospace and other fields due to its excellent physical and chemical properties. In the production and use of steel, due to equipment problems and consumption, defects such as scratches and cracks are prone to occur on the steel surface, which in turn affects product performance and even endangers life and property safety. Traditional defect detection mainly relies on manual visual inspection. This method is highly dependent on the workers' experience and physiological state, has high efficiency and low cost, and is subject to human subjectivity, and its accuracy is difficult to guarantee. At present, with the development of machine vision and deep learning, the automatic detection of surface defects of objects based on deep neural networks is efficient and low cost, and has important application value.

[0003] With the continuous change of technology, many excellent models have emerged. Among them, the YOLO series of algorithms are widely used in the field of real-time detection of industrial defects due to their light models and fast detection speed. In the YOLO series of algorithms, YOLOv8 is another major version after YOLOv5. The architecture of YOLO v8 is similar to that of v5, consisting of three parts: the backbone network, the feature enhancement network (neck) and the detection head (head). However, the C3 module of v5 is replaced by the convolution to fully connected (C2f) module, which enhances the feature extraction ability of small targets. At the same time, the update of head and anchorfree further enhances the performance of YOLO v8. Although YOLO v8 is already a relatively mature algorithm, some problems still occur in the actual application of steel surface defect detection. For example, for complex defects, the accuracy is not high; when the defect is similar to the background or not obvious, the target frame positioning will be inaccurate. Therefore, an accurate method for steel surface defect detection is needed. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method for training a steel surface defect detection model, which mainly solves the problem in the prior art of lacking an accurate steel surface defect detection method.

[0005] The objective of the present invention is achieved through the following solutions:

[0006] According to an embodiment of the present invention, a training method for a steel surface defect detection model is provided, comprising the following steps: S1, initializing the steel surface defect detection model, wherein the steel surface defect detection model is a YOLO v8 network, wherein a convolution to fully connected C2f module in a backbone network of the YOLO v8 network is encoded using dual branches; S2, constructing a training data set, wherein the training data set includes images of steels with different defect types and images of defect-free steels, wherein each image also includes a corresponding defect type as a label; S3, training the steel surface defect detection model based on the training data set, and updating the model parameters based on a preset loss function.

[0007] According to an embodiment of the present invention, in the first branch of the dual branches, dynamic convolution and phantom convolution are performed in parallel.

[0008] According to an embodiment of the present invention, the convolution kernel of the dynamic convolution is adjustable.

[0009] According to an embodiment of the present invention, the number of the convolution kernels is at least 2.

[0010] According to an embodiment of the present invention, in the second branch of the dual branches, the input feature map is segmented, and the output of each layer in the second branch is spliced ​​at the last layer.

[0011] According to an embodiment of the present invention, an attention layer is connected after the convolutional layer in the second branch.

[0012] According to an embodiment of the present invention, the attention layer adopts a channel attention mechanism and a spatial attention mechanism.

[0013] According to an embodiment of the present invention, when both the channel attention mechanism and the spatial attention mechanism are adopted, the order of the two is interchangeable.

[0014] According to an embodiment of the present invention, each convolutional layer adopts a residual connection.

[0015] According to an embodiment of the present invention, attention weighting is performed after the features of the two branches are fused.

[0016] Compared with the prior art, the present invention has the following beneficial effects: through dual-branch parallel encoding and final feature fusion, the feature extraction capability of the backbone network is improved and the accuracy of the defect detection model is enhanced; the introduction of dynamic convolution and phantom convolution reduces the computational burden and improves the model training efficiency; the residual structure better alleviates the problem of information loss and provides better feature information for subsequent defect identification, which also helps to improve the accuracy of the defect detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The embodiments of the present invention are further described below with reference to the accompanying drawings:

[0018] Figure 1 Schematic diagram of the flow of a method for training a steel surface defect detection model according to an embodiment of the present invention.

[0019] Figure 2 Schematic diagram of the composition of the C2f module of the steel surface defect detection model according to an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of the structure of a Conv-attention network in the C2f module of the steel surface defect detection model according to an embodiment of the present invention.

[0021] Figure 4 Schematic diagram of the structure of another Conv-attention network in the C2f module of the steel surface defect detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Before proceeding to the following detailed description, it may be advantageous to set forth the definitions of certain words and phrases used throughout this patent document. The terms "coupling", "connection" and their derivatives refer to any direct or indirect communication or connection between two or more elements, regardless of whether those elements are in physical contact with each other. The terms "transmission", "reception" and "communication" and their derivatives cover direct and indirect communication. The terms "include" and "comprising" and their derivatives refer to including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with..." and its derivatives refer to including, including within, interconnecting, containing, contained within, connecting or connecting with, coupling or coupling with, communicating with, cooperating, interweaving, parallel, close to, binding or binding with, having, having attributes, having a relationship or having a relationship with, etc. The term "controller" refers to any device, system or part thereof that controls at least one operation. Such a controller can be implemented with hardware, or a combination of hardware and software and / or firmware. The functions associated with any particular controller can be centralized or distributed, whether local or remote. The phrase "at least one of", when used with a list of items, means that different combinations of one or more of the listed items may be used, and only one of the items in the list may be required. For example, "at least one of A, B, C" includes any of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C.

[0023] Definitions for other specific words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior and future uses of such defined words and phrases.

[0024] In this patent document, the application combination of modules and the division level of sub-modules are only used for illustration, and the application combination of modules and the division level of sub-modules may have different forms without departing from the scope of the present disclosure.

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] As described in the background technology, the prior art lacks an accurate steel surface defect detection method. In view of the problem, as shown in the figure, according to an embodiment of the present invention, a training method for a steel surface defect detection model is proposed, comprising the following steps: S1, initializing the steel surface defect detection model, the steel surface defect detection model is a YOLO v8 network, wherein the convolution to fully connected C2f module in the backbone network of the YOLO v8 network is encoded using a dual branch; S2, constructing a training data set, the training data set includes images of steels of different defect types and images of defect-free steels, wherein each image also includes a corresponding defect type as a label; S3, training the steel surface defect detection model based on the training data set, and updating the model parameters based on a preset loss function. As is known to all, the backbone network of YOLO v8 includes four C2f modules. In an embodiment of the present invention, the four C2f modules are replaced by the C2f module (or C2f-enhance) as described in the embodiment of the present invention. Through the dual-branch parallel encoding and the final feature fusion method, the feature extraction capability of the backbone network is improved, and the overall feature expression capability of the network is also improved.

[0027] In order to dynamically and adaptively extract features and reduce the amount of computation, such as Figure 2 As shown, according to an embodiment of the present invention, in the first branch of the dual branches, dynamic convolution and phantom convolution are performed in parallel. In detail, according to an embodiment of the present invention, the convolution kernel of the dynamic convolution is adjustable. For example, according to an embodiment of the present invention, the number of convolution kernels is at least 2. In the case where dynamic convolution increases the amount of computation, in order to ensure a certain training efficiency, it is necessary to adopt technical means to reduce the amount of computation. As we all know, phantom convolution greatly reduces the amount of computation through linear operations. Among them, after experiments, in order to better achieve the effect of reducing the computational cost, the attention value should be between 0 and 1, and the phantom convolution should include at least one ordinary convolution layer. In addition, it is also shown that in the first branch (left branch), a layer of 1×1 ordinary convolution is first used to reduce the dimension of the feature map, increase the number of channels, and finally the two branches are feature fused through the add function.

[0028] In addition, in order to alleviate the problem of information loss and provide better feature information for subsequent defect identification, refer to Figure 2 According to an embodiment of the present invention, in the second branch of the dual branch, the input feature map is segmented, and the output of each layer in the second branch is spliced ​​in the last layer. By adopting the residual connection, the gradient can be transmitted more easily. During back propagation, due to the direct jump connection, the gradient will not disappear too quickly, making training easier. In addition, the residual connection allows information to flow directly from the input layer to the subsequent layers, alleviating the problem of difficult information transmission in deep networks, and helping to maintain a more direct flow of information. In addition, this also allows the construction of very deep networks without encountering the problem of gradient disappearance.

[0029] For details, refer again Figure 2 , shows that the Split layer is used to split the channels, and the number of channels is divided into two parts. One part passes through a module composed of n Conv-attention (or only Conv) connected in series, and each time it passes through a layer of Conv-attention, the output is saved in a tensor, which alleviates the problem of information loss. n can be modified as needed. Finally, the features of each layer are spliced ​​with the other part of the features after the split layer, and the feature map remains unchanged, and the number of channels is 2(n+1)c (where c is the number of channels).

[0030] In order to help the model focus on processing important information, thereby improving the performance and accuracy of the model, according to an embodiment of the present invention, an attention layer is connected after the convolution layer in the second branch. Figure 3 As shown, according to an embodiment of the present invention, the attention layer adopts a channel attention mechanism and a spatial attention mechanism. Two 3×3 convolutional layers with a step size of 1 are also shown, after which the SE channel attention mechanism and the STN spatial attention mechanism are added respectively. Among them, SE uses global average pooling to compress features in the spatial dimension to obtain a 1×1×C feature map (C is the number of channels), and then uses a fully connected layer to obtain channel attention information, and finally multiplies it with the original feature map matrix to obtain a feature map with enhanced channel information. The STN attention mechanism uses affine transformation to make the feature map have spatial invariance, which can better adapt to angle transformation and spatial transformation in the detection process.

[0031] In addition, according to different application scenarios, according to an embodiment of the present invention, when both the channel attention mechanism and the spatial attention mechanism are adopted, the order of the two mechanisms can be interchanged. Figure 3 As shown, the SE layer is set at the front layer of the module, and Figure 4 In the example, the STN layer is set at the front layer of the module. This order change also has a certain impact on the training efficiency of the model.

[0032] In addition, considering the trade-off between model accuracy and model operation speed, according to an embodiment of the present invention, each convolutional layer adopts a residual connection. If the residual connection is not used, the model operation speed can be accelerated; however, if the residual connection is used, the model accuracy can be improved.

[0033] In addition, in order to take into account the feature information of the left branch, according to an embodiment of the present invention, attention weighting is performed after the feature fusion of the two branches. In detail, an attention layer can be inserted after the final feature fusion add and before the next convolution layer. Similarly, a channel attention mechanism and a spatial attention mechanism can be used.

[0034] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0035] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0036] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for training a steel surface defect detection model, characterized in that: The following steps are involved: S1, initializing the steel surface defect detection model, the steel surface defect detection model is a YOLO v8 network, wherein the convolution to fully connected C2f module in the backbone network of the YOLO v8 network adopts dual branches for encoding; S2. Constructing a training data set, wherein the training data set includes images of steels with different defect types and images of defect-free steels, wherein each image also includes a corresponding defect type as a label; S3. Training the steel surface defect detection model based on the training data set, and updating the model parameters based on a preset loss function.

2. The method for training an image recognition model for detecting illegal items according to claim 1, characterized in that: In the first branch of the dual branches, dynamic convolution and phantom convolution are performed in parallel.

3. The method for training a steel surface defect detection model according to claim 2, characterized in that: The convolution kernel of the dynamic convolution is adjustable.

4. The method for training a steel surface defect detection model according to claim 3, characterized in that: The number of the convolution kernels is at least 2.

5. The method for training a steel surface defect detection model according to claim 1, characterized in that: In the second branch of the dual branch, the input feature map is segmented, and the output of each layer in the second branch is concatenated in the last layer.

6. A method for training a steel surface defect detection model according to claim 5, characterized in that: The convolutional layer in the second branch is followed by an attention layer.

7. The method for training a steel surface defect detection model according to claim 6, characterized in that: The attention layer adopts channel attention mechanism and spatial attention mechanism.

8. The method for training a steel surface defect detection model according to claim 7, characterized in that: When both the channel attention mechanism and the spatial attention mechanism are adopted, the order of the two is interchangeable.

9. The method for training a steel surface defect detection model according to claim 6, characterized in that: Each convolutional layer uses residual connections.

10. The method for training a steel surface defect detection model according to claim 1, characterized in that: Attention weighting is performed after the features of the two branches are fused.

Images