Steel surface defect detection method and system based on local-global context perception

Abstract

The present invention provides a steel surface defect detection method and system based on local-global context perception, which relates to the field of image processing technology. The method comprises: obtaining a steel image to be detected; inputting the steel image into a steel surface defect detection model for detection, and obtaining a steel surface defect detection result; a local-global perception network uses dilated convolutions with different dilation rates to extract local features of different scales, and when obtaining global features, the query vector, key vector and value vector are all input into a convolution module, a deep convolution module and a point-by-point convolution module for processing; a multi-level interactive network uses the outputs of different layers of the local-global perception network to perform cross-layer feature fusion; a reinforced recognition network fuses the output features of the multi-level interactive network, and enhances and detects the fused features to obtain a steel surface defect detection result. The present invention can improve the accuracy and efficiency of steel defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a steel surface defect detection method and system based on local-global context perception. Background Art

[0002] During the production process, steel may produce various surface defects such as scratches, cracks, inclusions, patches, etc. These defects not only affect the appearance of the product, but also reduce the overall performance of the product. Therefore, accurately identifying and removing these defects is crucial to ensure that the final product meets strict quality standards.

[0003] In steel defect detection, there are many types of steel defects, and each type has different characteristics and manifestations, which increases the difficulty of steel defect detection. Due to the significant differences in the size and shape of the target objects, and the many small and complex background defects in steel products, the existing detection models are insufficient in integrating the detailed features from the shallow layer of the network and the semantic features from the deep layer, and fail to fully utilize the feature information provided by each layer of the network, so the accuracy and efficiency of different types of defect detection are insufficient. Summary of the invention

[0004] In order to address the deficiencies of the prior art, the object of the present invention is to provide a steel surface defect detection method and system based on local-global context perception to improve the accuracy and efficiency of steel defect detection.

[0005] To achieve the above objectives, according to some embodiments, a first aspect of the present invention provides a steel surface defect detection method based on local-global context perception, comprising:

[0006] Acquire the steel image to be inspected;

[0007] Inputting the steel image to be inspected into the trained steel surface defect detection model for inspection to obtain the steel surface defect detection result;

[0008] Among them, the steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses void convolutions with different void rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results.

[0009] A second aspect of the present invention provides a steel surface defect detection system based on local-global context perception, comprising:

[0010] A data acquisition module is configured to acquire an image of steel to be inspected;

[0011] The defect detection module is configured to input the steel image to be detected into the trained steel surface defect detection model for detection, and obtain the steel surface defect detection result;

[0012] Among them, the steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses void convolutions with different void rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] The present invention provides a steel surface defect detection method and system based on local-global context perception, and designs a local-global perception network, a multi-level interactive network and an enhanced recognition network; the local-global perception network includes a local perception module and a context-global perception module, the local perception module uses a hole convolution with different hole rates to fully extract the edge, texture and other detail information of the steel image, and learns local features of different scales while expanding the receptive field; the context-global perception module captures the global context information of the steel image, and at the same time reduces the model parameter amount by performing convolution, deep convolution and point-by-point convolution processing on the query vector, key vector and value vector respectively. The efficiency of defect detection is improved; the multi-level interactive network introduces the output of all layers of the local overall perception network for cross-layer feature fusion, enriches the feature representation, and further enhances the ability to capture details of complex small-size targets; the enhanced recognition network includes a multi-branch enhancement module and a cascaded attention feature enhancement module. The multi-branch enhancement module utilizes the fusion of different branches to combine features of different spatial scales and multi-level semantic features to enrich the feature representation; the cascaded attention feature enhancement module optimizes the feature expression by screening and enhancing image features directly related to steel defects and reducing the impact of other interfering information, thereby improving the detection capability of steel defects.

[0015] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0017] Figure 1 is a flow chart of the method in Embodiment 1 of the present invention;

[0018] Figure 2 Schematic diagram of the local-holistic perception network;

[0019] Figure 3 Schematic diagram of the convolution downsampling module;

[0020] Figure 4 is a schematic diagram of a first local perception module;

[0021] Figure 5 is a schematic diagram of a first contextual overall perception module;

[0022] Figure 6 It is a schematic diagram of a multi-level interaction network;

[0023] Figure 7 It is a schematic diagram of a multi-branch reinforcement module;

[0024] Figure 8 Schematic diagram of the cascaded attention feature enhancement module. DETAILED DESCRIPTION

[0025] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0026] Embodiment 1

[0027] Embodiment 1 of the present invention provides a method for detecting steel surface defects based on local-global context perception, such as Figure 1-Figure 8 As shown, including:

[0028] Acquire the steel image to be inspected;

[0029] Inputting the steel image to be inspected into the trained steel surface defect detection model for inspection to obtain the steel surface defect detection result;

[0030] Among them, the steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses void convolutions with different void rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results.

[0031] Surface defect detection is a key technology in the field of industrial quality control, which aims to use computer and visual technology to identify and locate defects on the surface of objects. In order to improve the accuracy and efficiency of steel defect detection, this embodiment constructs a steel surface defect detection model, uses the collected steel images in the production line to train the model, and finally encapsulates and deploys the trained model to realize the detection of steel surface defects. The specific process includes:

[0032] Step S1, collecting steel images in the production line, annotating the collected steel images and constructing a data set, and dividing the data set into a training set, a test set, and a validation set;

[0033] Step S2, building a local overall perception network, a multi-level interactive network, and an enhanced recognition network, connecting the local overall perception network, the multi-level interactive network, and the enhanced recognition network in sequence to build a steel surface defect detection model;

[0034] Step S3, using the training set to train the constructed steel defect detection model, using the validation set to evaluate the trained model, and using the test set to test it, to obtain a trained steel surface defect detection model;

[0035] Step S4: encapsulate and deploy the trained steel surface defect detection model to detect the location and category of steel defects.

[0036] In step S1, a series of steel images are collected on the production line, and a corresponding annotation file is created for each image. The collected images are annotated using the image annotation software labelme to ensure that each image has a corresponding annotation file. The annotation content includes the types of steel defects, such as cracks, inclusions, patches, surface pitting, oxide layers, and scratches, and the specific locations of the steel defects in the image must also be marked. After the annotation is completed, the annotation file is paired with the corresponding image file to construct a complete data set. Subsequently, the constructed data set is segmented into training set, test set, and validation set in an 8:1:1 ratio for model training and testing. This process ensures the effective use of data and accurate evaluation of model performance.

[0037] In step S2, the steel surface defect detection model includes a local overall perception network, a multi-level interaction network, and an enhanced recognition network connected in sequence. Figure 2 As shown, it includes a convolution downsampling module, a first local perception module, a first context overall perception module, a second local perception module, a second context overall perception module, a third local perception module, and a third context overall perception module which are connected in sequence.

[0038] The convolutional downsampling module is as follows Figure 3 As shown in the figure, it includes two convolution modules with a convolution kernel size of 1×1, a deep convolution module with a convolution kernel size of 3×3, and a convolution module with a convolution kernel size of 3×3. Specifically, for the input steel image x, it first passes through a convolution module with a convolution kernel size of 3×3 and a stride of 2 to reduce the feature map size to half of the original size, and the output is the feature map , and then the feature map Input to the first convolution module with a convolution kernel size of 1×1, adjust the channel dimension, and output the feature map , the feature map Input to the deep convolution module with a convolution kernel size of 3×3 and a step size of 2 to reduce the size of the feature map, extract the surface defect features of steel and reduce the number of model parameters, and output the feature map , the feature map Input to the second convolution module with a convolution kernel size of 1×1, adjust the channel dimension, and output the feature map , and finally the feature map With feature map Perform element-by-element addition to reuse input features and output feature maps .

[0039] The structures of the first local perception module, the second local perception module, and the third local perception module are the same. The structure of the first local perception module is described below by taking the first local perception module as an example. Figure 4 As shown in the figure, the first local perception module is composed of a long-distance local feature extraction module and a residual channel attention module. The long-distance local feature extraction module includes a dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 1, a dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 3, a dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 5, a SiLU activation function, and a convolution module with a convolution kernel size of 1×1. The feature map output by the convolution downsampling module Input to the dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 1, and output the feature map , the feature map Input to the dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 3, and output the feature map , the feature map Input to the dilated convolution module with a convolution kernel size of 3×3 and a dilation rate of 5, and output the feature map , the feature map The SiLU activation function is used to prevent the model from having gradient vanishing problems during training, and the output is the feature map , the feature map , feature map and feature map The Concat function is used for concatenation operations, and dilated convolutions with different dilation rates are used to expand the model's receptive field and learn features of different scales, which is helpful for deep feature learning of the network and improves the model's generalization ability for different steel defect categories. The feature map is output. , the feature map With feature map Perform element-by-element addition operations, establish residual edges, reuse input features, and output feature maps , the feature map Input to the convolution module with a convolution kernel size of 1×1, reduce the number of feature map channels, and output the feature map .

[0040] The long-distance local feature extraction module can be expressed by the following formula:

[0041] ;

[0042] ;

[0043] ;

[0044] in, It is a dilated convolution module with a dilated rate of 1 and a convolution kernel size of 3×3. It is a dilated convolution module with a dilation rate of 3 and a convolution kernel size of 3×3. It is a dilated convolution module with a dilation rate of 5 and a convolution kernel size of 3×3. represents element-by-element addition, is a convolution module with a convolution kernel size of 1×1, Concat is a concatenation operation, F is the feature map input by the long-distance local feature extraction module, F1 is the feature map processed by the dilated convolution module with a dilation rate of 1, F2 is the feature map processed by the dilated convolution modules with dilation rates of 1 and 3, and F3 is the feature map output by the long-distance local feature extraction module.

[0045] The long-distance local feature extraction module connects in parallel the dilated convolution module branches with dilated rates of 1, 3, and 5, the dilated convolution module branches with dilated rates of 1 and 3, and the dilated convolution module branch with a dilated rate of 1. While expanding the receptive field of the model and learning features of different scales, it enhances the learning of deep features and improves the model's generalization ability for different types of steel defects, thereby extracting more refined features of the steel surface.

[0046] The residual channel attention module includes a convolution module with a convolution kernel size of 3×3, a convolution module with a convolution kernel size of 1×1, two global average pooling modules, SiLU and Sigmoid activation functions. Specifically, the feature map output by the first local perception module Input to the convolution module with a convolution kernel size of 3×3, and output the feature map , the feature map After SiLU activation function, the output is the feature map , the feature map After the global average pooling module, the output is the feature map , the feature map Input to the convolution module with a convolution kernel size of 1×1, and output the feature map , the feature map The input is passed through the Sigmoid activation function, the weight matrix of each channel is calculated, and the feature map is output , the feature map With feature map Perform element-by-element multiplication, reuse input features, enhance feature representation, and output feature maps .

[0047] The residual channel attention module can be expressed by the following formula:

[0048] ;

[0049] in, is a convolution module with a convolution kernel size of 1×1, SiLU represents the SiLU activation function, represents the Sigmoid activation function, AvgPool is the global average pooling operation, It represents element-by-element multiplication, P is the feature map of the residual channel attention module input, and X1 is the feature map of the residual channel attention module output.

[0050] The first context overall perception module, the second context overall perception module, and the third context overall perception module have the same structure. The first context overall perception module is taken as an example to illustrate the structure of the first context overall perception module. Figure 5 As shown in the figure, the first contextual overall perception module is composed of a lightweight overall perception module and a fine feature extraction module. The lightweight overall perception module includes four convolution modules with a convolution kernel size of 1×1, three deep convolution modules with a convolution kernel size of 3×3, three point-by-point convolution modules with a convolution kernel size of 1×1, a batch normalization layer and a Softmax activation function. Specifically, for the feature map output by the first local perception module , firstly transform the feature map After batch normalization, the output is the feature map , the feature map Through three convolution linear changes, we get the query vector, key vector and value vector. Then we perform convolution, depth convolution and point-by-point convolution on the query vector, key vector and value vector respectively to reduce the number of model parameters and improve the efficiency of steel defect detection. The query vector is input into the convolution module with a convolution kernel size of 1×1, and the output is the feature map. , the feature map Input to the deep convolution module with a convolution kernel size of 3×3, and output the feature map , the feature map Input to the point-by-point convolution module with a convolution kernel size of 1×1, and output the feature map ; Input the key vector to the convolution module with a convolution kernel size of 1×1, and output the feature map , the feature map Input to the deep convolution module with a convolution kernel size of 3×3, and output the feature map , the feature map Input to the point-by-point convolution module with a convolution kernel size of 1×1, and output the feature map ; Input the value vector into the convolution module with a convolution kernel size of 1×1, and output the feature map , the feature map Input to the deep convolution module with a convolution kernel size of 3×3, and output the feature map , the feature map Input to the point-by-point convolution module with a convolution kernel size of 1×1, and output the feature map Finally, the feature map With feature map Perform element-by-element multiplication to obtain the attention score and output the feature map , the feature map The feature map is obtained by Softmax normalization. , the feature map With feature map Perform element-by-element multiplication to obtain the weighted output feature map , the global features of the image are obtained, thereby improving the semantic relationship in the context of steel surface defect detection and enabling the model to fully understand the semantic information of the entire image. Input to the convolution module with a convolution kernel size of 1×1 to adjust the number of channels and output the feature map , the feature map With feature map Add element by element to establish the residual edge and output the feature map .

[0051] The lightweight overall perception module can be expressed by the following formula:

[0052] ;

[0053] ;

[0054] ;

[0055] ;

[0056] Among them, T is the feature map input by the lightweight overall perception module, is a convolution module with a convolution kernel size of 1×1. is a deep convolution module with a convolution kernel size of 3×3. is a point-by-point convolution module with a convolution kernel size of 1×1. represents element-wise multiplication, It represents element-by-element addition, Softmax represents the Softmax activation function, BN represents batch normalization, Q represents the query vector, K represents the key vector, V represents the value vector, S1 represents the feature map of the query vector after being processed by the convolution module, the depth convolution module and the point-by-point convolution module, S2 represents the feature map of the key vector after being processed by the convolution module, the depth convolution module and the point-by-point convolution module, S3 represents the feature map of the value vector after being processed by the convolution module, the depth convolution module and the point-by-point convolution module, and S4 represents the feature map output by the lightweight overall perception module.

[0057] The fine feature extraction module contains a batch normalization layer, four convolution modules with a convolution kernel size of 1×1, and a SiLU activation function. After the batch normalization layer, the output is the feature map , the feature map Through the left and right branches, more refined local features of steel defects are extracted to improve the model's adaptability to different types of defects. Input a convolution module with a convolution kernel size of 1×1 to the left branch, and output the feature map , the feature map is activated by SiLU function to enhance the model expression ability, and the feature map is output , the feature map After a convolution module with a convolution kernel size of 1×1, the output is the feature map ; The feature map The input is sent to the right branch, and after passing through a convolution module with a convolution kernel size of 1×1, the output is a feature map. , the feature map With feature map Perform element-by-element multiplication and output the feature map , two parallel convolution paths are used to extract more detailed features from steel defects, and the feature map Input to a convolution module with a convolution kernel size of 1×1, adjust the channel dimension, and output the feature map , the feature map With feature map Add element by element, establish residual edge to prevent gradient disappearance, and output the feature map .

[0058] The fine feature extraction module can be expressed by the following formula:

[0059] ;

[0060] ;

[0061] ;

[0062] in, is a convolution module with a convolution kernel size of 1×1. represents element-wise multiplication, represents element-by-element addition, Softmax represents the Softmax activation function, SiLU represents the SiLU activation function, BN represents batch normalization, S4 represents the feature map output by the lightweight overall perception module, S5 represents the feature map output by the left branch of the fine feature extraction module, S6 represents the feature map output by the right branch of the fine feature extraction module, and S7 represents the feature map output by the fine feature extraction module.

[0063] For the input steel image x, the feature map is obtained after the convolution downsampling module outputs , the feature map Input to the first local perception module and output the feature map , the feature map Input to the first context overall perception module, and output the feature map , the feature map Input to the second local perception module and output the feature map , the feature map Input to the second context overall perception module, and output the feature map , the feature map Input to the third local perception module and output the feature map , the feature map Input to the third contextual overall perception module, and output the feature map By setting up three perception modules and three contextual overall perception modules, the detailed information of the edges and textures of steel surface defects is fully extracted, and the global contextual information of steel images is captured, which improves the model's ability to detect different types of defects.

[0064] The feature maps output by each layer of the local holistic perception network are input into the multi-level interactive network for enhancement to enrich the feature representation. Specifically, the input of the local holistic perception network is the feature map , feature map , feature map and feature map ,like Figure 6 As shown, the multi-level interactive network consists of a first-level information interaction fusion unit, a second-level information interaction fusion unit, a third-level information interaction fusion unit, a fourth-level information interaction fusion unit, a fifth-level information interaction fusion unit, a sixth-level information interaction fusion unit, a seventh-level information interaction fusion unit, an eighth-level information interaction fusion unit, a ninth-level information interaction fusion unit, a tenth-level information interaction fusion unit, an eleventh-level information interaction fusion unit and a twelfth-level information interaction fusion unit.

[0065] The first-level information interaction fusion unit includes the convolutional layer, upsampling layer and C3 module of YOLOV5 network connected in sequence. Input to the first level information interaction fusion unit, and output the feature map .

[0066] The second-level information interaction fusion unit includes the convolutional layer, upsampling layer and C3 module of the YOLOV5 network connected in sequence. Input to the second-level information interaction fusion unit, and output the feature map .

[0067] The third-level information interaction fusion unit includes the convolutional layer, upsampling layer and C3 module of the YOLOV5 network connected in sequence. Input to the third-level information interaction fusion unit, and output the feature map .

[0068] The fourth level information interaction fusion unit includes the convolutional layer and the C3 module of the YOLOV5 network connected in sequence, which transforms the feature map Input to the fourth level information interaction fusion unit, and output the feature map .

[0069] The fifth level information interaction fusion unit includes the first local perception module and the C3 module of the YOLOV5 network connected in sequence, which transforms the feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input to the fifth level information interaction fusion unit, and output the feature map The first local perception module is used for downsampling to achieve the fusion of features of different scales. The first local perception module is used for downsampling to achieve downsampling while enriching the representation of features.

[0070] The sixth level information interaction fusion unit includes a C3 module that sequentially connects the first local perception module and the YOLOV5 network to transform the feature map , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input to the sixth level information interaction fusion unit, and output the feature map .

[0071] The seventh-level information interaction fusion unit is composed of the first local perception module and the C3 module of the YOLOV5 network in sequence. , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input to the seventh level information interaction fusion unit, and output the feature map .

[0072] The eighth-level information interaction fusion unit is composed of the convolutional layer and the C3 module of the YOLOV5 network in sequence. , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input to the eighth level information interaction fusion unit, and output the feature map .

[0073] The ninth level information interaction fusion unit is composed of the convolution layer, upsampling layer and C3 module of YOLOV5 network in sequence. , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input it to the ninth-level information interaction fusion unit and output the feature map f1.

[0074] The tenth-level information interaction fusion unit is composed of the convolution layer, upsampling layer and C3 module of YOLOV5 network in sequence. , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input it to the tenth-level information interaction fusion unit and output the feature map f2.

[0075] The eleventh-level information interaction fusion unit is composed of the convolution layer, the upsampling layer and the C3 module of the YOLOV5 network in sequence. , feature map and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input it to the eleventh-level information interaction fusion unit and output the feature map f3.

[0076] The twelfth-level information interaction fusion unit is composed of the convolutional layer and the C3 module of the YOLOV5 network in sequence. and feature map The concatenation operation is performed through the Concat function, and the feature map is output , the feature map Input it to the twelfth-level information interaction fusion unit and output the feature map f4.

[0077] The multi-level interactive network enriches the feature representation by introducing the output of all layers of the local overall perception network, further enhancing the ability to capture details of complex small-sized targets. In the second layer (fifth-level information interaction fusion unit, sixth-level information interaction fusion unit, seventh-level information interaction fusion unit, eighth-level information interaction fusion unit) and the third layer (ninth-level information interaction fusion unit, tenth-level information interaction fusion unit, eleventh-level information interaction fusion unit, twelfth-level information interaction fusion unit) of the fusion part, all use the fusion of adjacent layer features, and the third layer uses the fusion of adjacent layers while fusing cross-layer features, realizing the effective fusion of multi-scale features, thereby improving the ability to identify defects with significant size differences.

[0078] The feature maps f1, f2, f3, and f4 output by the multi-level interactive network are input into the enhanced recognition network for enhancement and defect recognition. The enhanced recognition network includes a multi-branch enhancement module, a cascaded attention feature enhancement module, and a detection head of the YOLOV5 network connected in sequence.

[0079] Multi-branch strengthening module such as Figure 7As shown in the figure, it contains four parallel branches. The feature map f1 is input to the first branch to reduce the size of the feature map and adjust the number of feature map channels. The first branch contains a parallel global maximum pooling module and a global average pooling module, as well as a convolution module with a convolution kernel size of 1×1 and a deep convolution module with a convolution kernel size of 3×3; the feature map f2 is input to the second branch, and the second branch contains a deep convolution module with a convolution kernel size of 3×3 to adjust the number of feature map channels; the feature map f3 is input to the third branch, the feature map is doubled and the number of feature map channels is adjusted. The third branch contains a double upsampling module. The fourth branch contains a four-fold upsampling module and a deep convolution module with a convolution kernel size of 3×3. The feature map f4 is input to the fourth branch, the feature map is upsampled twice and the number of feature map channels is adjusted. The fourth branch contains a four-fold upsampling module and a deep convolution module with a convolution kernel size of 3×3. The feature maps obtained by the four branches are then concatenated to fuse information of different spatial scales and semantic levels to enrich the contextual feature representation. Finally, the number of channels is adjusted and refined features are extracted through a convolution module with a convolution kernel size of 1×1, a deep convolution module with a convolution kernel size of 3×3, and a point-by-point convolution module with a convolution kernel size of 1×1. Specifically, feature map f1 is passed through the global maximum pooling module to obtain feature map f11. Feature map f1 is passed through the global average pooling module to effectively integrate global spatial information and capture defect targets of different scales to obtain feature map f12. Feature map f11 and feature map f12 are concatenated using the Concat function to obtain feature map f13. Feature map f13 is input into a convolution module with a convolution kernel size of 1×1 to reduce the number of channels and obtain feature map f14. Feature map f14 is input into a deep convolution module with a convolution kernel size of 3×3 and obtain feature map f15. Feature map f2 is input into a deep convolution module with a convolution kernel size of 3×3 and obtain feature map f21. Feature map f3 is input into a two-fold upsampling module to obtain feature map f31. Feature map f31 is input into a deep convolution module with a convolution kernel size of 3×3 and obtain feature map f32. The feature map f4 is input to the four-fold upsampling module, and the output is the feature map f41. The feature map f41 is input to the deep convolution module with a convolution kernel size of 3×3, and the output is the feature map f42. The feature maps f15, f21, f32, and f42 are concatenated using the Concat function to obtain the feature map f 11 , the feature map f 11 Input to the convolution module with a convolution kernel size of 1×1, and output the feature map f 12 , the feature map f 12 Input to the deep convolution module with a convolution kernel size of 3×3, and output the feature map f 13 , the feature map f 13Input into the point-by-point convolution module with a convolution kernel size of 1×1, and output the feature map t.

[0080] The multi-branch enhancement module uses four parallel branches to combine features of different spatial scales and multi-level semantic features to enrich feature representation. The first branch uses parallel global average pooling modules and global maximum pooling modules, as well as 3x3 convolution modules to downsample while effectively integrating global spatial information to capture defective targets of different scales. The second branch uses 3x3 convolution modules to extract features. The third and fourth branches extract fine-grained features while upsampling features. The fusion of four different branches combines features of different spatial scales and multi-level semantic features to enrich feature representation.

[0081] The multi-branch reinforcement module can be expressed by the following formula:

[0082] First, the output feature maps of the four parallel branches are obtained:

[0083] ;

[0084] ;

[0085] ;

[0086] ;

[0087] in, is a convolution module with a convolution kernel size of 1×1. is a deep convolution module with a convolution kernel size of 3×3, Concat is a concatenation operation, UP2 is a double upsampling, UP4 is a quadruple upsampling, MaxPool is the global maximum pooling, AvgPool is the global average pooling, f1, f2, f3 and f4 are the feature maps output by the ninth-level information interaction fusion unit, the tenth-level information interaction fusion unit, the eleventh-level information interaction fusion unit, and the twelfth-level information interaction fusion unit of the multi-level interactive network respectively; A1 is the feature map output by the first branch; A2 is the feature map output by the second branch; A3 is the feature map output by the third branch; A4 is the feature map output by the fourth branch.

[0088] Secondly, concatenation and convolution operations are used to fuse information of different spatial scales and semantic levels to enrich the contextual feature representation.

[0089] ;

[0090] in, is a convolution module with a convolution kernel size of 1×1. is a deep convolution module with a convolution kernel size of 3×3. It is a point-by-point convolution module with a convolution kernel size of 1×1, Concat is a concatenation operation, and A5 is the feature map output by the multi-branch enhancement module.

[0091] The output feature map t of the multi-branch enhancement module is further input into the cascaded attention feature enhancement module for feature enhancement. Figure 8 As shown in the figure, it includes a channel attention module and a spatial attention module. The channel attention module includes three convolution modules with a convolution kernel size of 1×1, a Softmax activation function, a SiLU activation function, and a batch normalization layer. The channel attention allows the network to focus on the most informative feature channels, and calculates the importance weight of each channel through global average pooling and 1×1 convolution operations. Specifically, first, the feature map t is input into the first convolution module with a convolution kernel size of 1×1, and the output is the feature map t1. The feature map t1 is subjected to the Softmax activation function to obtain the weight matrix. The obtained weight matrix is ​​multiplied element-by-element with the original input feature map t to obtain the attention feature map t2. The weights of the feature maps are redistributed to strengthen the informative features and suppress irrelevant features. The feature map t2 is input into the second convolution module with a convolution kernel size of 1×1, and the output is the feature map t3. The feature map t3 is subjected to the batch normalization layer and the SiLU activation function to speed up the convergence of the model, and the output is the feature map t4. The feature map t4 is input into the third convolution module with a convolution kernel size of 1×1, and the output is the feature map t5.

[0092] The channel attention module can be expressed by the following formula:

[0093] ;

[0094] Among them, U is the feature map input by the cascaded attention feature enhancement module, is a convolution module with a convolution kernel size of 1×1, SiLU represents the SiLU activation function, is element-by-element multiplication, BN is batch normalization, Softmax represents the Softmax activation function, and Q1 is the feature map output by the channel attention module.

[0095] The spatial attention module includes two convolution modules with a convolution kernel size of 1×1 and two deep convolution modules with a convolution kernel size of 3×3. The spatial attention module focuses on the spatial dimension, emphasizes the important spatial positions in the feature map, and helps the model identify the areas in the feature map that are most likely to contain steel defects. Specifically, first, the feature map t is input into the first convolution module with a convolution kernel size of 1×1, and the output is the feature map p1. The feature map p1 is input into the first deep convolution module with a convolution kernel size of 3×3, and the output is the feature map p2. The feature map p2 is input into the second deep convolution module with a convolution kernel size of 3×3, and the output is the feature map p3. The feature map p3 is input into the second convolution module with a convolution kernel size of 1×1, and the output is the feature map p4.

[0096] The spatial attention module can be expressed by the following formula:

[0097] ;

[0098] in, is a convolution module with a convolution kernel size of 1×1. It is a deep convolution module with a convolution kernel size of 3×3, and Q2 is the feature map output by the spatial attention module.

[0099] Subsequently, the feature map t5 output by the channel attention module and the feature map p4 output by the spatial attention module are added element by element, and the feature map v is output; the feature map v is activated by the SiLU function to prevent the gradient from disappearing, and the feature map v1 is output. The feature map v1 and the feature map t are reused by establishing residual edge input features, and the element-by-element multiplication operation is performed to output the feature map v2. The combination of channel attention and spatial attention significantly improves the model's sensitivity to steel defect features, thereby achieving more accurate defect detection in various complex backgrounds.

[0100] The cascaded attention feature enhancement module can be expressed by the following formula:

[0101] ;

[0102] Among them, SiLU represents the SiLU activation function, Q1 is the feature map output by the channel attention module, Q2 is the feature map output by the spatial attention module, is element-wise multiplication, It is element-wise addition.

[0103] The channel attention module of the cascaded attention feature enhancement module uses a 1×1 convolution module and an activation function to extract channel features without increasing the number of parameters. The spatial attention module uses a deep convolution module to extract and a 1×1 convolution module to enhance the features of defective areas, making the features of these areas more prominent. Finally, the spatial attention features are fused with the channel attention features, and the overall residual edge reuse input and output features are established. The cascaded attention feature enhancement module is designed to optimize the expression of features by screening and strengthening image features directly related to steel defects and reducing the influence of other interfering information, thereby improving the detection capability of steel defects.

[0104] The feature map v2 output by the cascaded attention feature enhancement module is input into the detection head of the YOLOV5 network of the enhanced recognition network to obtain the final feature image c, which contains the category, defect location and confidence information of the steel defect.

[0105] In step S3, the constructed steel defect detection model is trained by using the training set of the divided data set as input into the steel defect detection model.

[0106] During the training process, Adam was selected as the optimizer, the image size was initialized to 640×640, 300 rounds of training were performed, and 32 images were trained in each batch.

[0107] In view of the problem that the loss function in the prior art is not sensitive to the detection and identification of small steel target defects, the embodiment of the present invention designs an enhanced small target sensitive loss function to improve the identification and positioning of complex small targets in steel defects. Y-CIOU Used to improve the perception of complex small targets in steel defects, L Y-CIOU It is expressed by the following formula:

[0108] ;

[0109] ;

[0110] ;

[0111] Among them, IOU is the ratio of the intersection of the area of ​​the predicted box and the real box to the union of the area of ​​the predicted box and the real box. Represents the Euclidean distance between the center point of the predicted box and the true box, b, b gt are the center points of the predicted box and the true box, respectively. c represents the diagonal length of the minimum bounding box enclosed by the true box and the predicted box. w gt and h gt is the width and height of the real box, w and h are the width and height of the predicted box, cw 2 and 2are the width and height of the minimum bounding rectangle of the predicted box and the real box, respectively. A is the weight parameter, and B is the aspect ratio similarity. represents the Euclidean distance.

[0112] In addition, it also includes, through the formula Calculate the total loss function L Total , where L BCE is the cross entropy loss function.

[0113] In step S4, the optimal steel defect detection model training weights are exported from the training phase and saved. The optimal weights are then integrated into an inference script that is responsible for performing the model's prediction task. After completing this step, the inference script is embedded in the application. Finally, the application is deployed and runs on the local server, ensuring the automation and efficiency of steel surface defect detection.

[0114] Embodiment 2

[0115] This embodiment provides a steel surface defect detection system based on local-global context perception, including:

[0116] A data acquisition module is configured to acquire an image of steel to be inspected;

[0117] The defect detection module is configured to input the steel image to be detected into the trained steel surface defect detection model for detection, and obtain the steel surface defect detection result;

[0118] Among them, the steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses void convolutions with different void rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results.

[0119] It should be noted here that the various modules in this embodiment correspond one-to-one to the steps of the method in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A steel surface defect detection method based on local-global context perception, characterized in that: include: Acquire the steel image to be inspected; Inputting the steel image to be inspected into the trained steel surface defect detection model for inspection to obtain the steel surface defect detection result; The steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses dilated convolutions with different dilation rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results; The local overall perception network includes a convolution downsampling module, a first local perception module, a first context overall perception module, a second local perception module, a second context overall perception module, a third local perception module, and a third context overall perception module connected in sequence; The convolution downsampling module includes two convolution modules with a convolution kernel size of 1×1, a depth convolution module with a convolution kernel size of 3×3, and a convolution module with a convolution kernel size of 3×3; The first local perception module, the second local perception module, and the third local perception module all include a long-distance local feature extraction module and a residual channel attention module; The long-distance local feature extraction module sequentially passes the input feature map through the dilated convolution modules with a dilated rate of 1, a dilated rate of 3, and a dilated rate of 5, and processes it through the SiLU activation function to obtain a processed feature map, splices the processed feature map with the feature map output by the dilated convolution modules with a dilated rate of 1 and a dilated rate of 3, and adds the spliced ​​feature map to the feature map input by the long-distance local feature extraction module element by element, passes the added feature map through a convolution module with a convolution kernel size of 1×1 to reduce the number of channels, and obtains the output feature map of the long-distance local feature extraction module; The residual channel attention module sequentially passes the input feature map through a convolution module with a convolution kernel size of 3×3, a SiLU activation function, a global average pooling, a convolution module with a convolution kernel size of 1×1, and a Sigmoid activation function, and then multiplies the input feature map element by element with the feature map input by the residual channel attention module to obtain the feature map output by the residual channel attention module; The first context overall perception module, the second context overall perception module, and the third context overall perception module all include a lightweight overall perception module and a fine feature extraction module; The lightweight overall perception module obtains a query vector, a key vector and a value vector after batch normalization and convolution linear transformation of the input feature map, performs convolution, depth convolution and point-by-point convolution operations on the query vector, key vector and value vector respectively, obtains the global features of the input feature map according to the processed query vector, key vector and value vector, inputs the global features into a convolution module with a convolution kernel size of 1×1 to adjust the number of channels, and adds the global features to the feature map input by the lightweight overall perception module element by element to obtain the feature map output by the lightweight overall perception module; The fine feature extraction module, after batch normalization, inputs the input feature map into the left branch and the right branch respectively, wherein the feature map input into the left branch is processed by a convolution module with a convolution kernel size of 1×1, a SiLU activation function, and a convolution module with a convolution kernel size of 1×1 in sequence, and the feature map input into the right branch is processed by a convolution module with a convolution kernel size of 1×1, and the feature maps output by the left branch and the right branch are multiplied element by element and then input into a convolution module with a convolution kernel size of 1×1 to adjust the channel dimension, and are added element by element to the feature map input into the fine feature extraction module to obtain a feature map output by the fine feature extraction module; The multi-level interactive network includes a first-level information interaction fusion unit, a second-level information interaction fusion unit, a third-level information interaction fusion unit, a fourth-level information interaction fusion unit, a fifth-level information interaction fusion unit, a sixth-level information interaction fusion unit, a seventh-level information interaction fusion unit, an eighth-level information interaction fusion unit, a ninth-level information interaction fusion unit, a tenth-level information interaction fusion unit, an eleventh-level information interaction fusion unit, and a twelfth-level information interaction fusion unit; Among them, the feature map input by the fifth-level information interaction fusion unit is obtained by splicing the feature maps output by the third-level information interaction fusion unit and the fourth-level information interaction fusion unit; The feature map input by the sixth-level information interaction fusion unit is obtained by concatenating the feature maps output by the second-level information interaction fusion unit and the fourth-level information interaction fusion unit; The feature map input by the seventh-level information interaction fusion unit is obtained by concatenating the feature maps output by the first-level information interaction fusion unit and the third-level information interaction fusion unit; The feature map input by the eighth-level information interaction fusion unit is obtained by concatenating the feature maps output by the first-level information interaction fusion unit and the second-level information interaction fusion unit; The feature map input by the ninth-level information interaction fusion unit is obtained by concatenating the feature maps output by the seventh-level information interaction fusion unit and the eighth-level information interaction fusion unit; The feature map input by the tenth-level information interaction fusion unit is obtained by concatenating the feature maps output by the sixth-level information interaction fusion unit and the eighth-level information interaction fusion unit; The feature map input by the eleventh-level information interaction fusion unit is obtained by concatenating the feature maps output by the fifth-level information interaction fusion unit and the seventh-level information interaction fusion unit; The feature map input by the twelfth-level information interaction fusion unit is obtained by concatenating the feature maps output by the fifth-level information interaction fusion unit and the sixth-level information interaction fusion unit; The enhanced recognition network includes a multi-branch enhancement module, a cascaded attention feature enhancement module and a detection head of a YOLOV5 network connected in sequence; Among them, the multi-branch enhancement module includes a parallel first branch, a second branch, a third branch and a fourth branch. The first branch performs global maximum pooling and global average pooling on the feature map output by the twelfth-level information interaction fusion unit, and splices the results and inputs them into a convolution module with a convolution kernel size of 1×1 and a deep convolution module with a convolution kernel size of 3×3 for processing; the second branch inputs the feature map output by the eleventh-level information interaction fusion unit into a deep convolution module with a convolution kernel size of 3×3 for processing; the third branch inputs the feature map output by the tenth-level information interaction fusion unit into a deep convolution module with a convolution kernel size of 3×3 for processing. The feature map output by the first branch is upsampled twice and then input into a deep convolution module with a convolution kernel size of 3×3 for processing; the feature map output by the ninth-level information interaction fusion unit is upsampled four times by the fourth branch and then input into a deep convolution module with a convolution kernel size of 3×3 for processing; the feature maps output by the first branch, the second branch, the third branch and the fourth branch are concatenated and sequentially input into a convolution module with a convolution kernel size of 1×1, a deep convolution module with a convolution kernel size of 3×3, and a point-by-point convolution module with a convolution kernel size of 1×1 to obtain the feature map output by the multi-branch enhancement module; The cascaded attention feature enhancement module includes a channel attention module and a spatial attention module. The channel attention module multiplies the input feature map by the weight matrix obtained according to the input feature map element by element to obtain an attention feature map, and the attention feature map is processed by a convolution module with a convolution kernel size of 1×1, batch normalization and SiLU activation function, and a convolution module with a convolution kernel size of 1×1 in sequence to obtain a feature map output by the channel attention module; the spatial attention module processes the input feature map by a convolution module with a convolution kernel size of 1×1, two deep convolution modules with convolution kernel sizes of 3×3, and a convolution module with a convolution kernel size of 1×1 in sequence to obtain a feature map output by the spatial attention module; the feature map output by the channel attention module is added to the feature map output by the spatial attention module element by element, and after passing through the SiLU activation function, it is multiplied element by element with the feature map input by the cascaded attention feature enhancement module to obtain a feature map output by the cascaded attention feature enhancement module.

2. The steel surface defect detection method based on local-global context perception according to claim 1, characterized in that: The multi-level interactive network performs cross-layer fusion on the feature maps output by the convolution downsampling module, the first context overall perception module, the second context overall perception module, and the third context overall perception module.

3. The steel surface defect detection method based on local-global context perception according to claim 1, characterized in that: The first-level information interaction fusion unit, the second-level information interaction fusion unit, and the third-level information interaction fusion unit all include a convolutional layer, an upsampling layer, and a C3 module of a YOLOV5 network connected in sequence; The fourth-level information interaction fusion unit includes a convolutional layer and a C3 module of a YOLOV5 network connected in sequence; The fifth-level information interaction fusion unit, the sixth-level information interaction fusion unit, and the seventh-level information interaction fusion unit all include a first local perception module and a C3 module of a YOLOV5 network connected in sequence; The eighth-level information interaction fusion unit includes a convolutional layer and a C3 module of a YOLOV5 network connected in sequence; The ninth level information interaction fusion unit, the tenth level information interaction fusion unit, and the eleventh level information interaction fusion unit all include a convolutional layer, an upsampling layer, and a C3 module of a YOLOV5 network connected in sequence; The twelfth-level information interaction fusion unit includes a convolutional layer and a C3 module of a YOLOV5 network connected in sequence.

4. The steel surface defect detection method based on local-global context perception according to claim 1, characterized in that: The total loss function L during the steel surface defect detection model training process Total for: ； ； Among them, L BCE is the cross entropy loss function, L Y-CIOU To strengthen the small target sensitive loss function; IOU is the ratio of the intersection of the predicted box and the real box area to the union of the predicted box and the real box area. Represents the Euclidean distance between the center point of the predicted box and the true box, b, b gt are the center points of the predicted box and the true box respectively, c represents the diagonal length of the minimum bounding box enclosed by the true box and the predicted box, A is the weight parameter, and B is the aspect ratio similarity.

5. The steel surface defect detection system based on local-global context perception is characterized by: include: A data acquisition module is configured to acquire an image of steel to be inspected; The defect detection module is configured to input the steel image to be detected into the trained steel surface defect detection model for detection, and obtain the steel surface defect detection result; The steel surface defect detection model includes a local overall perception network, a multi-level interactive network and an enhanced recognition network connected in sequence. The local overall perception network uses dilated convolutions with different dilation rates to extract local features of different scales, and when using query vectors, key vectors and value vectors to obtain global features, the query vectors, key vectors and value vectors are all input into the convolution module, the deep convolution module and the point-by-point convolution module for processing; the multi-level interactive network uses the outputs of different layers of the local overall perception network to perform cross-layer feature fusion; the enhanced recognition network fuses the output features of the multi-level interactive network, enhances the fused features through the channel attention module and the spatial attention module, and inputs the enhanced features into the detection head to obtain the steel surface defect detection results; The local overall perception network includes a convolution downsampling module, a first local perception module, a first context overall perception module, a second local perception module, a second context overall perception module, a third local perception module, and a third context overall perception module connected in sequence; The convolution downsampling module includes two convolution modules with a convolution kernel size of 1×1, a depth convolution module with a convolution kernel size of 3×3, and a convolution module with a convolution kernel size of 3×3; The first local perception module, the second local perception module, and the third local perception module all include a long-distance local feature extraction module and a residual channel attention module; The long-distance local feature extraction module sequentially passes the input feature map through the dilated convolution modules with a dilated rate of 1, a dilated rate of 3, and a dilated rate of 5, and processes it through the SiLU activation function to obtain a processed feature map, splices the processed feature map with the feature map output by the dilated convolution modules with a dilated rate of 1 and a dilated rate of 3, and adds the spliced ​​feature map to the feature map input by the long-distance local feature extraction module element by element, passes the added feature map through a convolution module with a convolution kernel size of 1×1 to reduce the number of channels, and obtains the output feature map of the long-distance local feature extraction module; The residual channel attention module sequentially passes the input feature map through a convolution module with a convolution kernel size of 3×3, a SiLU activation function, a global average pooling, a convolution module with a convolution kernel size of 1×1, and a Sigmoid activation function, and then multiplies the input feature map element by element with the feature map input by the residual channel attention module to obtain the feature map output by the residual channel attention module; The first context overall perception module, the second context overall perception module, and the third context overall perception module all include a lightweight overall perception module and a fine feature extraction module; The lightweight overall perception module obtains a query vector, a key vector and a value vector after batch normalization and convolution linear transformation of the input feature map, performs convolution, depth convolution and point-by-point convolution operations on the query vector, key vector and value vector respectively, obtains the global features of the input feature map according to the processed query vector, key vector and value vector, inputs the global features into a convolution module with a convolution kernel size of 1×1 to adjust the number of channels, and adds the global features to the feature map input by the lightweight overall perception module element by element to obtain the feature map output by the lightweight overall perception module; The fine feature extraction module, after batch normalization, inputs the input feature map into the left branch and the right branch respectively, wherein the feature map input into the left branch is processed by a convolution module with a convolution kernel size of 1×1, a SiLU activation function, and a convolution module with a convolution kernel size of 1×1 in sequence, and the feature map input into the right branch is processed by a convolution module with a convolution kernel size of 1×1, and the feature maps output by the left branch and the right branch are multiplied element by element and then input into a convolution module with a convolution kernel size of 1×1 to adjust the channel dimension, and are added element by element to the feature map input into the fine feature extraction module to obtain a feature map output by the fine feature extraction module; The multi-level interactive network includes a first-level information interaction fusion unit, a second-level information interaction fusion unit, a third-level information interaction fusion unit, a fourth-level information interaction fusion unit, a fifth-level information interaction fusion unit, a sixth-level information interaction fusion unit, a seventh-level information interaction fusion unit, an eighth-level information interaction fusion unit, a ninth-level information interaction fusion unit, a tenth-level information interaction fusion unit, an eleventh-level information interaction fusion unit, and a twelfth-level information interaction fusion unit; Among them, the feature map input by the fifth-level information interaction fusion unit is obtained by splicing the feature maps output by the third-level information interaction fusion unit and the fourth-level information interaction fusion unit; The feature map input by the sixth-level information interaction fusion unit is obtained by concatenating the feature maps output by the second-level information interaction fusion unit and the fourth-level information interaction fusion unit; The feature map input by the seventh-level information interaction fusion unit is obtained by concatenating the feature maps output by the first-level information interaction fusion unit and the third-level information interaction fusion unit; The feature map input by the eighth-level information interaction fusion unit is obtained by concatenating the feature maps output by the first-level information interaction fusion unit and the second-level information interaction fusion unit; The feature map input by the ninth-level information interaction fusion unit is obtained by concatenating the feature maps output by the seventh-level information interaction fusion unit and the eighth-level information interaction fusion unit; The feature map input by the tenth-level information interaction fusion unit is obtained by concatenating the feature maps output by the sixth-level information interaction fusion unit and the eighth-level information interaction fusion unit; The feature map input by the eleventh-level information interaction fusion unit is obtained by concatenating the feature maps output by the fifth-level information interaction fusion unit and the seventh-level information interaction fusion unit; The feature map input by the twelfth-level information interaction fusion unit is obtained by concatenating the feature maps output by the fifth-level information interaction fusion unit and the sixth-level information interaction fusion unit; The enhanced recognition network includes a multi-branch enhancement module, a cascaded attention feature enhancement module and a detection head of a YOLOV5 network connected in sequence; Among them, the multi-branch enhancement module includes a parallel first branch, a second branch, a third branch and a fourth branch. The first branch performs global maximum pooling and global average pooling on the feature map output by the twelfth-level information interaction fusion unit, and splices the results and inputs them into a convolution module with a convolution kernel size of 1×1 and a deep convolution module with a convolution kernel size of 3×3 for processing; the second branch inputs the feature map output by the eleventh-level information interaction fusion unit into a deep convolution module with a convolution kernel size of 3×3 for processing; the third branch inputs the feature map output by the tenth-level information interaction fusion unit into a deep convolution module with a convolution kernel size of 3×3 for processing. The feature map output by the first branch is upsampled twice and then input into a deep convolution module with a convolution kernel size of 3×3 for processing; the feature map output by the ninth-level information interaction fusion unit is upsampled four times by the fourth branch and then input into a deep convolution module with a convolution kernel size of 3×3 for processing; the feature maps output by the first branch, the second branch, the third branch and the fourth branch are concatenated and sequentially input into a convolution module with a convolution kernel size of 1×1, a deep convolution module with a convolution kernel size of 3×3, and a point-by-point convolution module with a convolution kernel size of 1×1 to obtain the feature map output by the multi-branch enhancement module; The cascaded attention feature enhancement module includes a channel attention module and a spatial attention module. The channel attention module multiplies the input feature map by the weight matrix obtained according to the input feature map element by element to obtain an attention feature map, and the attention feature map is processed by a convolution module with a convolution kernel size of 1×1, batch normalization and SiLU activation function, and a convolution module with a convolution kernel size of 1×1 in sequence to obtain a feature map output by the channel attention module; the spatial attention module processes the input feature map by a convolution module with a convolution kernel size of 1×1, two deep convolution modules with convolution kernel sizes of 3×3, and a convolution module with a convolution kernel size of 1×1 in sequence to obtain a feature map output by the spatial attention module; the feature map output by the channel attention module is added to the feature map output by the spatial attention module element by element, and after passing through the SiLU activation function, it is multiplied element by element with the feature map input by the cascaded attention feature enhancement module to obtain a feature map output by the cascaded attention feature enhancement module.

Images