Steel structure surface corrosion detection method and system based on improved YOLOv8

Abstract

The invention discloses an improved YOLOv8-based steel structure surface corrosion detection method and system, and the method comprises the steps: employing a pre-trained improved YOLOv8 network model for a steel structure surface corrosion image to obtain a corrosion target detection result, and enabling the improved YOLOv8 network model to be based on a YOLOv8-OBB model, performing layer pruning on a deep convolutional layer of a backbone network of the YOLOv8-OBB model, introducing a context anchor point attention module CAA and a receptive field attention convolution RFAConv, and leading out shallow layer feature information of the backbone network to a neck network to realize feature fusion operation. The layer pruning refers to deletion of a convolutional layer of an eighth layer and a C2f layer of a ninth layer in the backbone network. The invention aims to solve the problems of low precision and excessive parameters of the existing corrosion target detection method, and realizes efficient, accurate and real-time steel structure surface corrosion target detection.

Description

Technical Field

[0001] The present invention relates to image processing technology in the field of steel structure engineering, and in particular to a method and system for detecting surface corrosion of steel structures based on improved YOLOv8. Background Art

[0002] During the service life of steel structure projects, temperature fluctuations, salty and humid environments, repeated loads and other factors may cause corrosion of steel structures, thus threatening the safety and stability of the entire structure. If not discovered and repaired in time, the service life of the steel structure will be seriously affected. Corrosion is a common form of damage to steel structures, and engineering accidents caused by corrosion occur from time to time. Therefore, it is necessary to focus on detecting steel structure corrosion during steel structure maintenance. There are currently several main methods for steel structure corrosion: (1) Traditional detection methods mainly rely on manual inspection or contact sensor detection and monitoring. Manual inspection operation space is limited, the observation and recording cycle is long, and data analysis relies on the subjective judgment of the inspectors; contact sensor detection requires a large number of contact sensors to be deployed, which is very expensive and easily affected by external environments such as temperature and humidity, causing distortion of the detection results. (2) The use of computer vision technology to detect steel structure corrosion mainly adopts the method of digital image processing (DIP). By analyzing the texture and color properties of different steel structure corrosion images, the steel structure corrosion damage can be identified. However, during the shooting process, there is often a lot of noise in the image background that affects the recognition and detection results. (3) Object detection models based on deep learning are gradually being used to solve complex image classification problems, and they all have good performance. However, there are still several problems with rust detection: 1) The rust targets in the image data are small, the background environment is complex, there is occlusion interference, and there are serious cases of false detection and missed detection; 2) Object detection models based on deep learning are relatively complex and have many parameters. The YOLOv8 model is a well-known object detection model based on deep learning, which mainly consists of three parts: the backbone network (Backbone), the neck network (Neck) and the detection head (Head). The YOLOv8-OBB (You Only LookOnce v8-Oriented Bounding Box) model is a variant of the YOLOv8 model. It introduces a version that supports oriented bounding boxes (OBB) based on YOLOv8, aiming to improve the detection accuracy of angled or rotated objects. The introduction of oriented bounding box (OBB) support specifically adds a track to the detection head (Head) for predicting angles and replaces the CIOU loss function with the ProbIOU loss function. However, both the YOLOv8 model and the YOLOv8-OBB model have the problems of complex results, numerous parameters, and deep convolution causing significant loss of feature information of small targets. Summary of the invention

[0003] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method and system for detecting surface corrosion of steel structures based on improved YOLOv8 are provided. The present invention aims to solve the problems of low precision and too many parameters in the existing corrosion target detection methods, and to achieve efficient, accurate and real-time target detection of surface corrosion of steel structures.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for detecting rust on the surface of a steel structure based on an improved YOLOv8 comprises the following steps: using a pre-trained improved YOLOv8 network model to obtain a rust target detection result for a rust image of the steel structure surface, wherein the improved YOLOv8 network model is obtained by pruning the deep convolutional layer of the backbone network of the YOLOv8-OBB model on the basis of the YOLOv8-OBB model, introducing a context anchor attention module CAA and a receptive field attention convolution RFAConv, and leading the shallow feature information of the backbone network to the neck network to realize feature fusion operation, wherein the deep convolutional layer of the backbone network of the YOLOv8-OBB model is pruned layer by layer. It refers to deleting the eighth convolution layer and the ninth C2f layer in the backbone network of the YOLOv8-OBB model, and introducing the contextual anchor attention module CAA and the receptive field attention convolution RFAConv, so that the backbone network in the improved YOLOv8 network model is a nine-layer structure consisting of sequentially connected convolution layer, convolution layer, C2f layer, receptive field attention convolution RFAConv, C2f layer C2f_RFAConv integrated with receptive field attention convolution, receptive field attention convolution RFAConv, C2f layer C2f_RFAConv integrated with receptive field attention convolution, contextual anchor attention module CAA and spatial pyramid pooling layer SPPF.

[0005] Optionally, the neck network in the improved YOLOv8 network model is composed of a fourteen-layer structure consisting of an upsampling module, a connection module, a context anchor attention module CAA, a C2f layer, an upsampling module, a connection module, a context anchor attention module CAA, a C2f layer, a convolutional layer, a connection module, a C2f layer, a convolutional layer, a connection module and a C2f layer, and the output features of the backbone network of the improved YOLOv8 network model are used as input features of the upsampling module of the first layer of the neck network and as input features of the connection module of the thirteenth layer of the neck network. The output features of the C2f layer of the fourth layer of the neck network are also used as the connection module of the tenth layer of the neck network. The output features of the C2f layers of the eighth, eleventh and fourteenth layers of the neck network are respectively used as the input features of the three detection heads in the improved YOLOv8 network model. The shallow feature information of the backbone network is led to the neck network to realize the feature fusion operation, which means that the output features of the C2f layer located in the third layer of the backbone network in the improved YOLOv8 network model are also used as the input features of the connection module located in the sixth layer of the neck network, and the output features of the C2f layer C2f_RFAConv integrated with the receptive field attention convolution located in the fifth layer of the backbone network in the improved YOLOv8 network model are also used as the input features of the connection module located in the second layer of the neck network.

[0006] Optionally, the contextual anchor attention module CAA processes input features in sequence, including a global average pooling operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, a strip convolution operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, and an activation function processing. The strip convolution operation includes a strip convolution operation with a convolution kernel size of 1×(11+2×n) and a strip convolution operation with a convolution kernel size of (11+2×n)×1, where n is a coefficient for adjusting the size of the strip convolution kernel.

[0007] Optionally, the size of the pooling window of the global average pooling operation is 7×7, the step size is 1, and the edge filling pixel size is 3.

[0008] Optionally, the C2f layer C2f_RFAConv that incorporates the receptive field attention convolution includes a convolutional layer, a slicing layer, an n-level bottleneck module, a connection module and a convolutional layer that are connected in sequence, wherein the output features of the slicing layer and the n-level bottleneck module are also used as input features of the connection module to achieve feature fusion of the output features of the slicing layer and the n-level bottleneck module through the connection module, and the bottleneck module includes a convolutional layer, a receptive field attention convolution RFAConv and a summation module that are connected in sequence, and the summation module is used to sum the input features of the bottleneck module and the output features of the receptive field attention convolution RFAConv as the output features of the bottleneck module.

[0009] Optionally, the processing of the input features by the receptive field attention convolution RFAConv includes: generating three spatial receptive field feature maps of size 9C×H×W through a 3×3 grouped convolution operation on an input feature map of size C×H×W, where C is the number of channels of the input feature map, and H and W are the height and width of the input feature map, respectively; generating a feature map of size C×H×W through average pooling AvgPool on the input feature map of size C×H×W, and after grouping into three groups, activating them through a Softmax activation function respectively to generate three attention maps of different sizes of 9C×H×W; the three spatial receptive field feature maps of size 9C×H×W and the three attention maps of different sizes of 9C×H×W are resized to C×3H×3W, and then a convolution operation with a convolution kernel size of 3×3 and a step size of 1 is performed to obtain a final output feature map of size C×H×W.

[0010] Optionally, the convolution layer is composed of a two-dimensional convolution operation, a batch normalization operation and a SiLU activation function; the three detection heads in the improved YOLOv8 network model are respectively used to detect rusted targets of different sizes on the surface of steel structures, each detection head includes three detection branches for respectively realizing bounding box prediction, category prediction and angle prediction, and each detection branch is composed of two convolution layers and a two-dimensional convolution module connected in sequence.

[0011] In addition, the present invention also provides a steel structure surface corrosion detection system based on improved YOLOv8, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8.

[0012] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 through a processor.

[0014] Compared with the prior art, the present invention mainly has the following advantages: 1. Aiming at the problem that the scale of rusted targets varies and the background environment of images is complex, the present invention introduces a contextual anchor attention module CAA to improve the YOLOv8 network model to capture remote context information, and enhances the expression ability of central features in the feature extraction process, thereby improving the ability of the YOLOv8 model to handle large-scale and long-shaped targets; 2. Aiming at the problem that the rusted targets in the pictures taken by drones are small, the background environment is complex, and the false detection and missed detection are serious, the present invention introduces the receptive field attention convolution RFAConv to improve the YOLOv8 network model, which can improve the YOLOv8 model's ability to process image details and retain more small target feature information; 3. In view of the problem that the YOLOv8-OBB model is relatively complex, has many parameters and deep convolution will cause a large loss of feature information of small targets, the present invention prunes the deep convolution layer of the backbone network of the YOLOv8-OBB model, deletes the eighth convolution layer and the ninth C2f layer in the backbone network of the YOLOv8-OBB model, and leads the shallow feature information of the backbone network to the neck network to realize feature fusion operation, which can reduce the algorithm space complexity of the YOLOv8-OBB model and improve the detection accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0016] Figure 2 Schematic diagram of the network structure of the improved YOLOv8 network model in an embodiment of the present invention.

[0017] Figure 3 Schematic diagram of the network structure of the context anchor attention module CAA in an embodiment of the present invention.

[0018] Figure 4 Schematic diagram of the network structure of the C2f layer C2f_RFAConv incorporating the receptive field attention convolution in an embodiment of the present invention.

[0019] Figure 5 Schematic diagram of the network structure of the receptive field attention convolution RFAConv in an embodiment of the present invention.

[0020] Figure 6 Schematic diagram of the network structure of the convolutional layer in an embodiment of the present invention.

[0021] Figure 7 Schematic diagram of the network structure of the spatial pyramid pooling layer SPPF in an embodiment of the present invention.

[0022] Figure 8 Schematic diagram of the network structure of the detection head in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0024] like Figure 1 As shown, the steel structure surface corrosion detection method based on the improved YOLOv8 in this embodiment includes the following steps: using the pre-trained improved YOLOv8 network model to obtain the corrosion target detection result of the steel structure surface corrosion image, the improved YOLOv8 network model is based on the YOLOv8-OBB model, the deep convolution layer of the backbone network of the YOLOv8-OBB model is pruned, and the context anchor attention module CAA and the receptive field attention convolution RFAConv are introduced, and the shallow feature information of the backbone network is led to the neck network to realize the feature fusion operation, the deep convolution layer of the backbone network of the YOLOv8-OBB model is pruned, and the eighth convolution layer and the ninth C2f layer of the backbone network of the YOLOv8-OBB model are deleted, and the context anchor attention module CAA (Context AnchorAttention) and the receptive field attention convolution RFAConv (Receptive-Field Attention Convolution), so that the backbone network in the improved YOLOv8 network model is a nine-layer structure consisting of sequentially connected convolution layer, convolution layer, C2f layer, receptive field attention convolution RFAConv, C2f layer C2f_RFAConv integrated with receptive field attention convolution, receptive field attention convolution RFAConv, C2f layer C2f_RFAConv integrated with receptive field attention convolution, contextual anchor attention module CAA and spatial pyramid pooling layer SPPF, as shown in the following figure. Figure 2 As an optional implementation, the steel structure surface corrosion image in this embodiment is a steel structure surface corrosion image collected by a drone. In addition, other methods can also be used to collect steel structure surface corrosion images.

[0025] Layer pruning is the operation of removing an entire layer (such as a convolutional layer or a fully connected layer) from a neural network. The rusted targets in the drone pictures are relatively small, and the YOLOv8-OBB model is relatively complex and has many parameters. Too deep a convolution will cause a large loss of feature information of small targets. In this embodiment, the YOLOv8-OBB model is pruned by layer, and the eighth convolutional layer and the ninth C2f layer in the backbone network of the YOLOv8-OBB model are deleted to reduce the algorithm space complexity and improve the detection accuracy of small targets. At the same time, the shallow feature information is fused in the Neck part, and the shallow feature information of the backbone network is led to the neck network to realize the feature fusion operation (the C2f layer located in the third layer of the backbone network and the upsampling layer feature information located in the fifth layer of the neck network are spliced ​​in the channel dimension), further enhancing the detection capability of small targets.

[0026] In this embodiment, the introduction of the contextual anchor attention module CAA refers to introducing the contextual anchor attention module CAA in the previous layer of the spatial pyramid pooling layer SPPF of the backbone network. The fixed pooling operation of the spatial pyramid pooling layer SPPF may confuse the features of the target and the background, while the contextual anchor attention module CAA can use global average pooling and strip convolution to capture the relationship between distant pixels and enhance the features in the central area, so as to better distinguish the target and the background and improve the model's recognition ability for difficult-to-detect targets.

[0027] In this embodiment, the introduction of receptive field attention convolution RFAConv refers to replacing the convolution layers of the 4th and 6th layers, and the C2f layers of the 5th and 7th layers in the backbone network with receptive field attention convolution RFAConv and C2f layers C2f_RFAConv integrated with receptive field attention convolution respectively. The receptive field of ordinary convolution layers is fixed and cannot adapt to the size of different targets. The receptive field attention convolution RFAConv uses the attention mechanism to dynamically allocate weights of different receptive fields, allowing the network to adaptively adjust the focus area in different scenarios. The C2f layer was originally used to fuse shallow and deep features. After being replaced with the C2f layer C2f_RFAConv integrated with the receptive field attention convolution, the feature fusion module will have the ability to dynamically adjust the receptive field, retain more small target feature information, emphasize the characteristics of the target area, and suppress background noise.

[0028] The backbone network in the final improved YOLOv8 network model is a nine-layer structure consisting of a sequentially connected convolution layer, a convolution layer, a C2f layer, a receptive field attention convolution RFAConv, a C2f layer C2f_RFAConv integrated with the receptive field attention convolution, a receptive field attention convolution RFAConv, a C2f layer C2f_RFAConv integrated with the receptive field attention convolution, a contextual anchor attention module CAA, and a spatial pyramid pooling layer SPPF. As an optional implementation, the output channels of the convolution layers located in the first and second layers of the backbone network of this embodiment are 32 and 64 respectively, the convolution kernel size is 3, and the step size is 2; the output channel of the C2f layer located in the third layer is 64. ; The output channel of the receptive field attention convolution RFAConv at the 4th layer is 128, the convolution kernel size is 3, and the stride is 2; the output channel of the C2f layer C2f_RFAConv with receptive field attention convolution at the 5th layer is 128; the output channel of the receptive field attention convolution RFAConv at the 6th layer is 256, the convolution kernel size is 3, and the stride is 2; the output channel of the C2f layer C2f_RFAConv with receptive field attention convolution at the 7th layer is 256; the output channel of the contextual anchor attention module CAA at the 8th layer is 256; the output channel of the spatial pyramid pooling layer SPPF at the 9th layer is 256, and the kernel size of the pooling convolution kernel is 5.

[0029] The C2f layer C2f_RFAConv that incorporates receptive field attention convolution replaces the second convolution layer of the bottleneck module with receptive field attention convolution RFAConv based on the C2f layer.

[0030] like Figure 2As shown in the figure, the neck network in the improved YOLOv8 network model consists of a fourteen-layer structure consisting of an upsampling module, a connection module, a context anchor attention module CAA, a C2f layer, an upsampling module, a connection module, a context anchor attention module CAA, a C2f layer, a convolutional layer, a connection module, a C2f layer, a convolutional layer, a connection module and a C2f layer. The output features of the backbone network of the improved YOLOv8 network model are used as the input features of the upsampling module of the first layer of the neck network and as the input features of the connection module of the thirteenth layer of the neck network. The output features of the C2f layer of the fourth layer of the neck network are also used as the input features of the connection module of the tenth layer of the neck network. Input features, the output features of the C2f layers of the eighth, eleventh and fourteenth layers of the neck network are respectively used as the input features of the three detection heads in the improved YOLOv8 network model. The shallow feature information of the backbone network is introduced to the neck network to realize the feature fusion operation, which means that the output features of the C2f layer located in the third layer of the backbone network in the improved YOLOv8 network model are also used as the input features of the connection module located in the sixth layer of the neck network, and the output features of the C2f layer C2f_RFAConv integrated with the receptive field attention convolution located in the fifth layer of the backbone network in the improved YOLOv8 network model are also used as the input features of the connection module located in the second layer of the neck network. The specific location of adding the context anchor attention module CAA to the neck network part is in the latter layer of the upsampling module. The upsampling operation of the upsampling module adopts the nearest neighbor interpolation algorithm, which may introduce unnecessary background noise or information redundancy, and does not distinguish the importance of different targets, and may not be able to highlight the key target area well. The context anchor attention module CAA can assign higher weights to the target area, while suppressing background noise, so that the high-resolution feature map is more focused on meaningful targets.As an optional implementation, in the neck network of this embodiment, the upsampling layer (Upsample) at the first layer in the neck network uses the nearest neighbor interpolation to change the length and width of the picture to twice the original; the connection module at the second layer is used to splice the output of the fifth layer with the output of the 10th layer at this layer; the output channel of the context anchor attention module CAA at the third layer is 384; the output channel of the C2f layer at the fourth layer is 128; the length and width of the picture of the upsampling layer using the nearest neighbor interpolation at the fifth layer is twice the original; the connection module at the sixth layer is used to splice the output of the third layer with the output of the 14th layer at this layer; the upper and lower modules at the seventh layer are used to splice the output of the third layer with the output of the 14th layer at this layer. The output channel of the text anchor attention module CAA is 192; the output channel of the C2f layer at the 8th layer is 64; the output channel of the convolution layer at the 9th layer is 128, the convolution kernel size is 3, and the step size is 2; the connection module at the 10th layer is used to splice the output of the 13th layer with the output of the 18th layer at this layer; the output channel of the C2f layer at the 11th layer is 128; the output channel of the convolution layer at the 12th layer is 64, the convolution kernel size is 3, and the step size is 2; the connection module at the 13th layer is used to splice the output of the 9th layer with the output of the 22nd layer at this layer; the output channel of the C2f layer at the 14th layer is 256.

[0031] The contextual anchor attention module CAA is used to capture long-range context information and enhance the expressiveness of central features during feature extraction, and performs well when dealing with large-scale or long-shaped targets. Figure 3 As shown, the processing of input features by the contextual anchor attention module CAA in this embodiment includes, in sequence, a global average pooling operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, a strip convolution operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, and an activation function processing. The strip convolution operation includes, in sequence, a strip convolution operation with a convolution kernel size of 1×(11+2×n) and a strip convolution operation with a convolution kernel size of (11+2×n)×1, where n is a coefficient for adjusting the size of the strip convolution kernel, and in this embodiment, the specific value is 0.

[0032] In this embodiment, the pooling window size of the global average pooling operation of the contextual anchor attention module CAA is 7×7, the step size is 1, and the edge filling pixel size is 3.

[0033] like Figure 4As shown, the C2f layer C2f_RFAConv that incorporates the receptive field attention convolution includes a convolutional layer, a slicing layer, an n-level bottleneck module, a connection module and a convolutional layer that are connected in sequence, wherein the output features of the slicing layer and the n-level bottleneck module are also simultaneously used as input features of the connection module to achieve feature fusion of the output features of the slicing layer and the n-level bottleneck module through the connection module, and the bottleneck module includes a convolutional layer, a receptive field attention convolution RFAConv and a summation module that are connected in sequence, and the summation module is used to sum the input features of the bottleneck module and the output features of the receptive field attention convolution RFAConv as the output features of the bottleneck module.

[0034] The main idea of ​​the receptive field attention convolution RFAConv is to combine the spatial attention mechanism with the convolution operation to optimize the working mode of the convolution kernel. Learning the attention map through the interactive receptive field feature information can improve the network and assign different weights to each part of the receptive field to emphasize the important areas and weaken the relatively unimportant areas. Figure 5 As shown in the figure, the processing of the input features by the receptive field attention convolution RFAConv includes: generating three spatial receptive field feature maps of size 9C×H×W through a 3×3 grouped convolution operation on the input feature map of size C×H×W, where C is the number of channels of the input feature map, and H and W are the height and width of the input feature map respectively; generating a feature map of size C×H×W through average pooling AvgPool on the input feature map of size C×H×W, and after grouping into three groups, activating them through the Softmax activation function respectively to generate three attention maps of different sizes of 9C×H×W; the three spatial receptive field feature maps of size 9C×H×W and the three attention maps of different sizes of 9C×H×W are resized to C×3H×3W, and then through a convolution operation with a convolution kernel size of 3×3 and a step size of 1, the final output feature map of size C×H×W is obtained.

[0035] The contextual anchor attention module CAA combines convolution operations with strip convolution operations to promote adaptive feature extraction of local and global context information, and improves the ability of the YOLOv8 model to handle large-scale and long-shaped targets. Taking the contextual anchor attention module CAA of the 8th layer as an example, the output feature map of the C2f layer C2f_RFAConv with the 7th layer integrated into the receptive field attention convolution Input the context anchor attention module CAA, first perform average pooling operation to obtain local area features , which can be expressed as: , in, Represents the input feature map Perform average pooling operation. Indicates that the window size is 7, Indicates that the step size is 1, Indicates that the number of padding for the input feature map is 3. Local area features The convolution operation with a convolution kernel of 1×1, the strip convolution operation with a convolution kernel of 1×11, the strip convolution operation with a convolution kernel of 11×1, and the convolution operation with a convolution kernel of 1×1 are performed in sequence. Finally, the feature map of the contextual anchor attention module CAA of the 8th layer is output through the sigmoid activation function. , the processing process is expressed as: , , , , , , in, , and Represents the output feature map, weights and bias of the i-th convolution operation in the contextual anchor attention module CAA, i=1,2,3,4; express Activation function.

[0036] The C2f (Coarse-to-Fine) layer is a neural network structure used to process information from coarse to fine levels. It usually contains multiple levels of processing units that can gradually extract and fuse features of different scales, thereby achieving information integration from coarse to fine. The processing flow of each C2f layer is the same. Taking the C2f layer of the third layer of the backbone network as an example, the output of the second convolutional layer is Direct Transfer And the convolution processing part It is expressed as: , in, Indicates direct transfer of the part without convolution operation; Indicates the part processed by the Bottleneck module; , and Respectively and The number of channels, represents the branch processing of the C2f layer. The neck network contains two convolutional layers. The feature map obtained by two convolutional layers is added to the original feature map to form a residual enhanced feature map. The processing process can be expressed as: , in, and They represent the weight and bias of the first convolutional layer in the C2f layer of the third layer of the backbone network, and They represent the weight and bias of the second convolutional layer in the C2f layer of the third layer respectively.

[0037] After processing and Perform channel dimension splicing to obtain the spliced ​​feature map , and finally concatenate the feature maps After the final convolutional layer, the output of the C2f layer of the third layer is obtained. , the processing process is expressed as: , , in, represents the concatenation operation in the channel dimension, and denote the weights and biases of the final convolutional layer in the C2f layer of the second layer, respectively.

[0038] The processing flow of each receptive field attention convolution RFAConv is the same. Taking the receptive field attention convolution RFACONV of the 4th layer as an example, the output of the 3rd layer C2f layer is , the attention map is obtained by average pooling operation, 1×1 group convolution operation, and softmax activation function. ; Output of the third layer C2f layer The receptive field spatial features are obtained by sequentially performing 3×3 group convolution operations, normalization operations, and ReLU activation functions. ; Finally, , Multiply and output feature map The processing process is expressed as: , , , in, Indicates size The grouped convolution, stands for batch normalization, and There are two activation functions. is a set of vectors, is a vector No. item, To get the maximum value. After a convolution operation with a convolution kernel of 3×3 and a step size of 3, the output of the receptive field attention convolution RFAConv of the 4th layer is obtained : , in, and They represent the weight and bias of the final convolution operation in the receptive field attention convolution RFACONV of the 4th layer, respectively. Indicates the size of the convolution kernel.

[0039] The convolution layer is a basic module commonly used in convolutional neural networks (CNNs), which is mainly used to extract local spatial information from input features. Figure 6 As shown in Figure 1, the convolution layer consists of a two-dimensional convolution operation, a batch normalization operation, and a SiLU activation function. The processing flow of each convolution layer is the same. Taking the first Conv layer as an example, the input image (the rust image of the steel structure surface) is represented as , , and Respectively represent the number of channels, height and width of the input image, represents the set of real numbers. The processing can be expressed as: , , in, It is the feature map output after the first Conv layer; is an activation function; represents the convolution operation; and are the weights and biases of the first Conv layer; Indicates the step size of each slide during the convolution operation.

[0040] The spatial pyramid pooling layer SPPF is an improved spatial pyramid pooling technology, which is mainly used for multi-scale feature extraction to enhance the model's ability to detect targets of different sizes. The spatial pyramid pooling layer SPPF can significantly expand the receptive field by combining convolution operations and maximum pooling operations, helping the steel structure surface corrosion recognition model to more accurately identify small target groups. The output feature map of the contextual anchor attention module CAA at the 8th layer is After inputting the spatial pyramid pooling layer SPPF, Figure 7 As shown in Figure 2, the spatial pyramid pooling layer SPPF first passes through a convolutional layer with a step size of 1 to extract preliminary features and adjust the number of channels to obtain an activated feature map. , and then activate the feature map The maximum pooling operation is performed to further improve the receptive field and obtain the pooling feature map. The processing process is expressed as: , , in, Represents the feature map generated from pooling convolution kernels of different sizes in the pooling layer, that is, the pooling feature map; Represents the activation feature map Perform multi-scale maximum pooling operations. Specifically, after three maximum pooling operations and skip connections, the feature map is finally activated. It is spliced ​​with the pooled feature map in the channel dimension, and the feature information of different scales is fused through convolution operation, while the number of channels is compressed to generate the final output feature map. , which can be expressed as: , in, and They represent the weights and biases of the final convolutional layer in the SPFF layer, respectively.

[0041] In this embodiment, the three detection heads in the improved YOLOv8 network model are used to detect rusted targets of different sizes on the surface of steel structures. The first detection head uses the output of the eighth layer of the neck network (a feature map of size 160×160×64) as input, the second detection head uses the output of the eleventh layer of the neck network (a feature map of size 80×80×128) as input, and the third detection head uses the output of the last layer of the neck network (a feature map of size 40×40×256) as input.

[0042] like Figure 8 As shown, in this embodiment, each detection head includes three detection branches for realizing bounding box prediction, category prediction and angle prediction respectively. Each detection branch is composed of two convolution layers and a two-dimensional convolution module connected in sequence. The first detection branch is used for bounding box prediction, the second detection branch is used for category prediction, and the third detection branch is used for predicting angle. The three detection branches have the same structure, but use different loss functions. In the figure, BBox Loss is the bounding box loss, and Cls Loss is the classification loss. Each detection branch can be expressed as: , , , in, ( , and ), ( , and )and ( , and ) represents the output feature map, weight and bias of the i-th layer convolution operation of a detection branch in the detection head, i=1,2,3.

[0043] like Figure 2 As shown, after the improved YOLOv8 network model is constructed, the mapping relationship between the steel structure surface corrosion image and the corrosion target detection result can be established by training the improved YOLOv8 network model. The steel structure surface corrosion image to be detected is input into the improved YOLOv8 network model to obtain the corresponding corrosion target detection result. In this embodiment, the training of the improved YOLOv8 network model includes: Step 1: Dataset construction. The DJI M350RTK drone is equipped with a Zenmuse P1 camera, and flies along a predetermined path around the steel structure building to cover the surface of the building for data collection. The rust targets in the collected pictures are annotated by the labelimg2 software, and the 4056×3040 high-resolution drone images are cropped into 640×640 pixel images, totaling 2077 images, which are divided into training set and verification set in a ratio of 7:3. Step 2: Train the improved YOLOv8-OBB model to finally obtain a trained improved YOLOv8-OBB model. Input the rust image of the steel structure surface into the trained improved YOLOv8 network model to obtain the rust target detection result.

[0044] In order to objectively evaluate the detection performance of the improved YOLOv8 network model in this embodiment, the following indicators are used in this embodiment: Frames per second (FPS), which indicates the number of frames transmitted per second. FLOPs, which indicates the number of floating-point operations required for the model to make predictions, reflects the computational complexity of the model. Precision (Precision, P), which indicates the proportion of samples that are truly positive among the samples identified as positive by the model. Recall (Recall, R), which indicates the ratio of the number of samples correctly identified as positive by the model to the total number of positive samples. Mean Average Precision (mAP), mAP is the average of various average precisions (AP), AP is the area under the precision and recall curves, and the function expression is: , , , , , Among them, N represents the number of detected images, T represents the time consumed for detection, TP represents the number of images correctly detected as shedding diseases, FP represents the number of images incorrectly classified as diseases, and FN represents the number of images that are actually diseases but not detected. C Indicates the total number of categories. The ablation experiment results of the improved YOLOv8-OBB model in this embodiment are shown in Tables 1 and 2.

[0045] Table 1 Ablation experiment scheme for improved YOLOv8-OBB model

[0046] Table 2 Ablation experiment results of improved YOLOv8-OBB model

[0047] In Tables 1 and 2, Net-1 represents the layer pruning operation and shallow feature information fusion operation in the YOLOv8-OBB model. Compared with the YOLOv8-OBB model, the accuracy P is increased by 1.4 percentage points, R is increased by 2.2 percentage points, mAP is increased by 0.6 percentage points, the number of parameters is reduced by 68.4%, the number of floating-point operations is basically the same, and the FPS is increased by 5.75 frames / s. Net-2 introduces the contextual anchor attention module CAA on the basis of Net-1. Compared with the Net-1 model, the accuracy P is increased by 0.9 percentage points, R is reduced by 0.6 percentage points, the number of parameters, mAP and floating-point operations are basically the same, and the FPS is reduced by 16.56 frames / s. It can be seen that the receptive field attention convolution RFAConv can improve the detection effect without almost increasing the computational cost. Net-3 introduces the receptive field attention convolution RFAConv on the basis of Net-1. Compared with the Net-1 model, the accuracy P is basically the same, R is increased by 0.9 percentage points, and mAP is increased by 0.8 percentage points, but the number of parameters is increased by 18%, the number of floating-point operations is increased by 9.1×109 times, and the FPS is reduced by 17.47 frames / s. This is because the context anchor attention module CAA is introduced, and new calculation operations and parameters are added to dynamically model the dependency between the target and the context. Net-4 is the improved YOLOv8-OBB model of this embodiment. Compared with the YOLOv8-OBB model, the accuracy P is increased by 3.8 percentage points, R is increased by 5 percentage points, mAP is increased by 3.5 percentage points, the number of parameters is reduced by 63.2%, the number of floating-point operations is increased by 11.1×109 billion times, and the FPS is reduced by 17.47 frames / s. It can be seen that the above ablation experiment proves that the improved YOLOv8-OBB model of this embodiment can effectively improve the detection accuracy of steel structure surface corrosion and the spatial complexity of the model.

[0048] In addition, this embodiment also provides a steel structure surface corrosion detection system based on improved YOLOv8, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8.

[0049] In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 through a processor.

[0050] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 through a processor.

[0051] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present invention may be in the form of methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that instructions executed by the processor of a computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0052] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting surface corrosion of steel structures based on improved YOLOv8, characterized in that: The method comprises the following steps: using a pre-trained improved YOLOv8 network model to obtain a rust target detection result for a steel structure surface rust image, wherein the improved YOLOv8 network model is based on a YOLOv8-OBB model, and the deep convolutional layer of the backbone network of the YOLOv8-OBB model is pruned, and a context anchor attention module CAA and a receptive field attention convolution RFAConv are introduced, and the shallow feature information of the backbone network is led to the neck network to realize feature fusion operation, and the deep convolutional layer of the backbone network of the YOLOv8-OBB model is pruned by layer, and the deep convolutional layer of the backbone network of the YOLOv8-OBB model is deleted. The eighth convolution layer and the ninth C2f layer in the backbone network of the B model, and by introducing the contextual anchor attention module CAA and the receptive field attention convolution RFAConv, the backbone network in the improved YOLOv8 network model is a nine-layer structure consisting of sequentially connected convolution layers, convolution layers, C2f layers, receptive field attention convolution RFAConv, C2f layers integrated with receptive field attention convolution C2f_RFAConv, receptive field attention convolution RFAConv, C2f layers integrated with receptive field attention convolution C2f_RFAConv, contextual anchor attention modules CAA and spatial pyramid pooling layers SPPF.

2. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 1, characterized in that: The neck network in the improved YOLOv8 network model is composed of a fourteen-layer structure consisting of an upsampling module, a connection module, a contextual anchor attention module CAA, a C2f layer, an upsampling module, a connection module, a contextual anchor attention module CAA, a C2f layer, a convolutional layer, a connection module, a C2f layer, a convolutional layer, a connection module and a C2f layer, wherein the output features of the backbone network of the improved YOLOv8 network model are used as the input features of the upsampling module of the first layer of the neck network and as the input features of the connection module of the thirteenth layer of the neck network, and the output features of the C2f layer of the fourth layer of the neck network are used as the input features of the connection module of the tenth layer of the neck network. The output features of the C2f layers of the eighth, eleventh and fourteenth layers of the neck network are respectively used as input features of the three detection heads in the improved YOLOv8 network model. The shallow feature information of the backbone network is led to the neck network to realize the feature fusion operation, which means that the output features of the C2f layer of the third layer of the backbone network in the improved YOLOv8 network model are also used as input features of the connection module of the sixth layer of the neck network, and the output features of the C2f layer C2f_RFAConv integrated with the receptive field attention convolution located in the fifth layer of the backbone network in the improved YOLOv8 network model are also used as input features of the connection module of the second layer of the neck network.

3. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 1 or 2, characterized in that: The contextual anchor attention module CAA processes input features in sequence, including a global average pooling operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, a strip convolution operation, a two-dimensional convolution operation with a convolution kernel size of 1×1, and an activation function processing. The strip convolution operation includes a strip convolution operation with a convolution kernel size of 1×(11+2×n) and a strip convolution operation with a convolution kernel size of (11+2×n)×1, where n is a coefficient for adjusting the size of the strip convolution kernel.

4. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 3 is characterized in that: The size of the pooling window of the global average pooling operation is 7×7, the step size is 1, and the edge filling pixel size is 3.

5. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 1, characterized in that: The C2f layer C2f_RFAConv that incorporates the receptive field attention convolution includes a convolutional layer, a slicing layer, an n-level bottleneck module, a connection module and a convolutional layer that are connected in sequence, wherein the output features of the slicing layer and the n-level bottleneck module are also simultaneously used as input features of the connection module to achieve feature fusion of the output features of the slicing layer and the n-level bottleneck module through the connection module, and the bottleneck module includes a convolutional layer, a receptive field attention convolution RFAConv and a summation module that are connected in sequence, and the summation module is used to sum the input features of the bottleneck module and the output features of the receptive field attention convolution RFAConv as the output features of the bottleneck module.

6. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 1, characterized in that: The processing of the receptive field attention convolution RFAConv for input features includes: generating three spatial receptive field feature maps of size 9C×H×W through a 3×3 grouped convolution operation on an input feature map of size C×H×W, where C is the number of channels of the input feature map, and H and W are the height and width of the input feature map, respectively; generating a feature map of size C×H×W through average pooling AvgPool on the input feature map of size C×H×W, and after grouping into three groups, respectively activating them through a Softmax activation function to generate three attention maps of different sizes of 9C×H×W; the three spatial receptive field feature maps of size 9C×H×W and the three attention maps of different sizes of 9C×H×W are resized to C×3H×3W, and then a convolution operation with a convolution kernel size of 3×3 and a step size of 1 is performed to obtain a final output feature map of size C×H×W.

7. The method for detecting surface corrosion of steel structures based on improved YOLOv8 according to claim 2, characterized in that: The convolution layer is composed of a two-dimensional convolution operation, a batch normalization operation and a SiLU activation function; the three detection heads in the improved YOLOv8 network model are respectively used to detect rusted targets of different sizes on the surface of steel structures, each detection head includes three detection branches for respectively realizing bounding box prediction, category prediction and angle prediction, and each detection branch is composed of two convolution layers and a two-dimensional convolution module connected in sequence.

8. A steel structure surface corrosion detection system based on improved YOLOv8, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 as described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the steel structure surface corrosion detection method based on improved YOLOv8 as described in any one of claims 1 to 7 through a processor.

Images