Surface defect small target detection method based on multi-scale feature interaction

Abstract

The invention discloses a surface defect small target detection method based on multi-scale feature interaction. Comprising the steps that a surface image, collected in real time, of a to-be-detected product is input into a detection model, and after the detection model processes the surface image, an image marked with possible defect classification categories, bounding box coordinates and defect target existence confidence is output; wherein the detection model is a model taking a YOLOv11 network as a basic structure and comprises a backbone network, a neck network and a detection head which are sequentially connected in series; a multi-scale feature extraction SMK module and an attention mechanism SELA-S module are introduced into the backbone network; two feature fusion Fault modules are introduced into the neck network. According to the method, the multi-scale target features under the complex background can be effectively extracted and fused, and the method has good processing performance for distinguishing defects in the picture from the complex background and multi-scale target detection. And in a surface defect detection task, the method can be accurately focused on a defect area.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and particularly relates to a small target detection method for surface defects based on multi-scale feature interaction. Background Art

[0002] Traditional steel defect detection methods mainly include manual visual inspection and contact detection techniques. However, these traditional methods have many limitations. Manual inspection is not only inefficient but also affected by human factors, prone to misjudgment and missed detection. Contact detection can detect some hidden defects, but it is complex to operate, has a long detection cycle, and often requires pretreatment of the steel surface, which may even cause damage to the steel. In addition, these traditional methods are limited by labor and cannot meet the requirements of long-term and high-intensity work, and cannot meet the high-precision and high-efficiency needs in large-scale production, that is, the assembly line process.

[0003] Compared with traditional methods, machine learning methods have higher efficiency and stability in steel defect detection, can automatically analyze large-scale data and reduce human errors. However, this method still has some limitations. First, traditional machine learning methods usually rely on manually designed features, but it is difficult to effectively extract key features when facing defects with irregular shapes and blurred boundaries, resulting in missed detection or misjudgment. Second, machine learning methods usually have poor environmental adaptability and are difficult to adapt to the environment of steel manufacturing factories. Moreover, some machine learning models consume a large amount of computing resources, have limited real-time performance, and are difficult to meet the requirements of practical applications.

[0004] In recent years, deep learning, especially convolutional neural network (CNN), has shown excellent performance in steel surface defect detection. Traditional detection methods, including manual detection methods and machine learning methods, cannot meet the high-precision and high-efficiency requirements, and the introduction of deep learning technology has greatly promoted the development of steel surface defect detection. In particular, the benchmark model YOLOv11 algorithm selected in this paper has become a popular choice in the industry due to its high processing speed and high detection accuracy. However, YOLOv11 still has missed detection or misjudgment in the identification of some complex or subtle defects in the application of steel surface defect detection. Summary of the Invention

[0005] In order to solve the technical problem of easy missed detection or misjudgment in the identification of complex or subtle defects in current surface defect detection, the present invention provides a small target detection method for steel surface defects based on multi-scale feature interaction, which can effectively improve the detection accuracy of steel surface defects during the steel production process.

[0006] In order to achieve the above technical purpose, the technical solution of the present invention is:

[0007] A small target detection method for surface defects based on multi-scale feature interaction, comprising the following steps:

[0008] Step 1, input the surface image of the product to be detected collected in real time into the detection model;

[0009] Step 2, after the detection model processes the surface image, output an image annotating the possible defect classification categories, bounding box coordinates, and the confidence level of the existence of defect targets;

[0010] Among them, the detection model is a model based on the YOLOv11 network as the basic structure, including a backbone network, a neck network, and a detection head connected in series in sequence;

[0011] The multi-scale feature extraction SMK module and the attention mechanism SELA-S module are introduced into the backbone network; two feature fusion Fafusion modules are introduced into the neck network;

[0012] The SMK module is arranged between the original third and fourth C3K2 modules of the backbone network, and includes a residual structure path, a local information extraction path, and a spatial information extraction path for respectively processing the feature maps input into the SMK module. The processing results of the three paths are spliced and used as the output;

[0013] The SELA-S module is arranged between the original SPPF module and the C2PSA module of the backbone network, and includes a spatial attention mechanism path and a channel attention mechanism path for respectively processing the feature maps input into the SELA-S module. The processing results of the two paths are respectively multiplied by the input feature map, and the multiplied results are added to the input feature map and used as the output;

[0014] The two Fafusion modules are connected in series. The first Fafusion module is arranged after the first upsampling module of the neck network, and the output is processed by the first C3K2 module and the second upsampling module of the neck network and used as the input of the second Fafusion module; both Fafusion modules include four paths for processing feature maps, and the processing results of the four paths are spliced and the channels are adjusted and used as the output.

[0015] Further, in the said Step 1, the detection model is obtained through the following steps of training:

[0016] Step 101, obtain the surface images of products with surface defects, annotate them as a dataset, and then perform data augmentation on the images in the dataset to expand the dataset;

[0017] Step 102: Divide the augmented dataset into a training set, a validation set, and a test set according to a preset ratio. Then input the training set into the detection model for iterative training, and regularly input the validation set into the detection model during the iterative training process to adjust the model training strategy according to the validation effect. Finally, after the iterative training is completed, input the test set to verify the training effect and evaluate the performance of the detection model.

[0018] Further, in step 101, the data augmentation is to perform operations on the image including flipping, rotation, cropping, and brightness adjustment, so as to generate a new processed image based on the original image to expand the dataset.

[0019] Further, in step 102, during the iterative training process, multi-scale features are extracted through the backbone network, then cross-layer feature interaction and fusion are realized by the neck network, and finally the detection head outputs the classification category, bounding box coordinates, and target existence confidence as the results. Then compare the results with the label values and perform backpropagation to update the parameters of the detection model. At the same time, regularly input the validation set into the detection model for verification to further adjust the parameters of the detection model according to the verification results. Finally, after verifying the training effect through the test set and evaluating the performance of the detection model, save the parameters of the trained detection model.

[0020] Further, in step 102, during the training process, the detection head uses Focaler-PIoUv2 generated by combining the Focaler-IoU loss function and the PIoUv2 loss function as the bounding box loss function;

[0021] The loss of Focaler-IoU is expressed as:

[0022] ;

[0023] While represents the reconstructed value of Focaler-IoU, and the expression is:

[0024] ;

[0025] Where represents the intersection over union of the predicted box and the ground truth box; and are two threshold parameters, and , ∈ [0, 1];

[0026] PIoUv2 is obtained by combining the PIoUv1 loss function and a non-monotonic attention function controlled by a single hyperparameter The expression of PIoUv1 is:

[0027] ;

[0028] represents the natural constant; represents the penalty factor, and the expression is:

[0029] ;

[0030] where is the distance between the left edge of the predicted box and the left edge of the ground truth box, is the distance between the right edge of the predicted box and the right edge of the ground truth box, is the distance between the top edge of the predicted box and the top edge of the ground truth box, is the distance between the bottom edge of the predicted box and the bottom edge of the ground truth box, is the width of the ground truth box, is the height of the ground truth box;

[0031] Rather than the non - monotonic attention function The expression is:

[0032] ;

[0033] where represents the input value of the non - monotonic attention function;

[0034] Therefore, the loss of PIoUv2 is expressed as:

[0035] ;

[0036] where represents the hyperparameter that controls the behavior of the attention function; represents a parameter used to measure the quality of the anchor box, ;

[0037] In summary, the loss of Focaler - PIoUv2 The expression is:

[0038] .

[0039] Furthermore, the backbone network of the detection model includes twelve sequentially connected module structures. The first layer is the first Conv module, the second layer is the second Conv module, the third layer is the first C3K2 module, the fourth layer is the third Conv module, the fifth layer is the second C3K2 module, the sixth layer is the fourth Conv module, the seventh layer is the third C3K2 module, the eighth layer is the SMK module, the ninth layer is the fourth C3K2 module, the tenth layer is the SPPF module, the eleventh layer is the SELA-S attention mechanism module, and the twelfth layer is the C2PSA module.

[0040] Furthermore, the SMK module includes:

[0041] The residual structure path includes a 1×1 Conv module;

[0042] The local information extraction path includes a sequentially connected Conv module and a channel attention mechanism module;

[0043] The spatial information extraction path includes two sub-paths of different scales. One includes a sequentially connected 1×1 Conv module, 3×3 Conv module, dilated convolution module with a dilation of 2 for the x direction, and dilated convolution module with a dilation of 2 for the y direction. The other includes a sequentially connected 1×1 Conv module, 5×5 Conv module, dilated convolution module with a dilation of 3 for the x direction, and dilated convolution module with a dilation of 3 for the y direction;

[0044] The Concat module for splicing the residual structure path, local information extraction path, and spatial information extraction path.

[0045] Furthermore, the SELA-S attention mechanism module includes:

[0046] The spatial attention mechanism path includes two sequentially connected groups of spatial feature weight extraction modules. Each group of modules includes a sequentially connected average pooling layer, feature extraction layer, group normalization, and Sigmoid function. The average pooling layer of one group is the vertical average pooling layer, and the other is the horizontal average pooling layer. The output of the first group of modules is multiplied by the feature map input to the SELA-S attention mechanism module and then input to the second group of modules to obtain spatial feature weights in two different directions;

[0047] The channel attention mechanism path includes a sequentially connected adaptive pooling layer and convolutional layer, and finally passes through the sigmoid function to obtain channel feature weights;

[0048] Then, the processing results of the two paths, including the spatial feature weights in the vertical and horizontal directions and the channel feature weights obtained by the channel attention mechanism path, are multiplied by the original feature map respectively. Finally, each result is added to the original feature map serving as the residual branch to obtain the output.

[0049] Further, the neck network of the detection model includes a twelve-layer module structure connected in series in sequence. The first layer is the first upsampling module, the second layer is the first Fafusion module, the third layer is the first C3K2 module, the fourth layer is the second upsampling module, the fifth layer is the second Fafusion module, the sixth layer is the second C3K2 module, the seventh layer is the first Conv module, the eighth layer is the first Concat module, the ninth layer is the third C3K2 module, the tenth layer is the second Conv module, the eleventh layer is the second Concat module, and the twelfth layer is the fourth C3K2 module.

[0050] The input end of the first Fafusion module is also respectively connected to the second C3K2 module, the third C3K2 module, and the fourth C3K2 module of the backbone network; the input end of the second Fafusion module is also respectively connected to the first C3K2 module, the second C3K2 module, and the third C3K2 module of the backbone network; the output end of the second C3K2 module is also connected to the detection head; the input end of the first Concat module is also connected to the first C3K2 module of the neck network; the output end of the third C3K2 module is also connected to the detection head; the input end of the second Concat module is also connected to the C2PSA module of the backbone network.

[0051] Further, in the Fafusion module in the neck network, the four paths include: the downsampling path of the large-size feature map, including DepthwiseConv and PointwiseConv connected in series in sequence; the path for adjusting the number of channels of the medium-size feature map, including a Conv module for adjusting the number of channels of the feature map; two paths for small-size feature maps, both including an Unsample module. Finally, the output pictures of the four paths are concatenated, and then the channels are adjusted through Conv to obtain the output feature map.

[0052] The technical effects of the present invention are as follows:

[0053] Based on the existing YOLOv11 network, the present invention introduces the SMK module to replace the Conv module in the original network model, which enables the model to effectively extract and fuse multi-scale target features in complex backgrounds. At the same time, the number of parameters and computational complexity are lower than those of the Conv module, and it has good processing performance for distinguishing defects from complex backgrounds and multi-scale target detection in steel pictures.

[0054] At the same time, the present invention also introduces the SELA-S attention mechanism, enabling the model to more precisely focus on the defect area in the steel surface defect detection task, while suppressing irrelevant background information, enhancing the feature expression ability, and thus improving the detection accuracy and generalization ability.

[0055] The present invention also enhances the information interaction ability of the network at different scale feature levels through the multi-scale feature fusion pyramid network and the feature fusion module Fafusion, ensuring the global information sharing between different scale features of the network, and the formed model has a better detection effect.

[0056] When training the model, by using the Focaler-PIoUv2 bounding box loss function, the model can focus on different regression samples in different detection tasks, and at the same time ensure that the model has more accurate and stable bounding box regression.

[0057] Moreover, when obtaining the dataset for training, data augmentation is used to preprocess the dataset, effectively expanding the morphological diversity of defect samples. This data augmentation mechanism alleviates the overfitting risk under the condition of limited samples, thereby improving the generalization ability of the model. Brief Description of the Drawings

[0058] Figure 1 Schematic diagram of the multi-scale feature interaction steel surface defect small target detection model for the technical solution of the present invention.

[0059] Figure 2 Schematic diagram of the SMK module in the technical solution of the present invention.

[0060] Figure 3 Schematic diagram of the Fafusion module in the technical solution of the present invention.

[0061] Figure 4 Schematic diagram of the SELA-S module in the technical solution of the present invention. Detailed Embodiments

[0062] The technical solution of the present invention will be further described below in conjunction with the drawings and specific embodiments.

[0063] Embodiment 1:

[0064] As Figure 1 shown, a detection method based on the multi-scale feature interaction steel surface defect small target detection model provided in this embodiment includes:

[0065] Input the surface image of the product to be detected collected in real time into the detection model.

[0066] The detection model is a model based on the YOLOv11 network as the basic structure, including a backbone network, a neck network, and a detection head connected in series in sequence. In this embodiment, a multi-scale feature extraction module SMK module and a SELA-S attention mechanism module are introduced into the YOLOv11 network, and then a multi-scale feature fusion pyramid network and a feature fusion module Fafusion are introduced, so as to obtain the multi-scale feature interaction steel surface defect small target detection model used in this embodiment. The model structure is as Figure 2 shown.

[0067] In the backbone network of this embodiment, a multi-scale feature extraction SMK module and an attention mechanism SELA-S module are introduced. The backbone network includes a twelve-layer module structure connected in series in sequence, where the first layer is the first Conv module, the second layer is the second Conv module, the third layer is the first C3K2 module, the fourth layer is the third Conv module, the fifth layer is the second C3K2 module, the sixth layer is the fourth Conv module, the seventh layer is the third C3K2 module, the eighth layer is the SMK module, the ninth layer is the fourth C3K2 module, the tenth layer is the SPPF module, the eleventh layer is the SELA-S attention mechanism module, and the twelfth layer is the C2PSA module.

[0068] In this embodiment, the Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series in sequence.

[0069] The C3K2 module of this embodiment includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

[0070] Traditional ordinary Conv only uses a fixed convolutional kernel size (such as 3×3) and can only perceive features of a fixed size, and can only learn patterns under single-scale features, which may ignore some key defect information, resulting in the model being difficult to capture both fine scratches and large-area defects at the same time. Therefore, the SMK module is introduced in this embodiment. The SMK module in this embodiment refers to the Scale Multi-path Kernel fusion module. The SMK module is set between the original third and fourth C3K2 modules of the backbone network, and includes a residual structure path, a local information extraction path, and a spatial information extraction path for separately processing the feature maps input into the SMK module. The processing results of the three paths are concatenated and used as the output.

[0071] See Figure 3, the SMK module extracts multi-scale features from images, processes different scale features differently, and finally fuses the obtained multi-scale features, enhancing the model's multi-scale feature extraction and fusion capabilities to effectively enhance the perception ability of multi-scale defect targets under complex backgrounds. The SMK module improves the detection accuracy of steel surface defects through multi-scale convolution combinations, while reducing the computational complexity of the model, enabling it to have higher computational efficiency while ensuring detection performance. In addition, the SMK module can fully distinguish defect regions from complex background information, improving the robustness of the model to defect targets of different scales, thus showing more excellent adaptability in steel defect detection tasks.

[0072] Among the three main paths of the SMK module, the residual structure path includes a 1×1 Conv module. The local information extraction path includes a Conv module and a channel attention mechanism module connected in series. The spatial information extraction path includes two sub-paths of different scales. One includes a 1×1 Conv module, a 3×3 Conv module, a dilated convolution module with a dilation of 2 for the x direction and a dilated convolution module with a dilation of 2 for the y direction connected in series. The other includes a 1×1 Conv module, a 5×5 Conv module, a dilated convolution module with a dilation of 3 for the x direction, and a dilated convolution module with a dilation of 3 for the y direction connected in series. At the same time, there is a Concat module at the end of the SMK module for splicing the residual structure path, the local information extraction path, and the spatial information extraction path. The final output feature map is obtained by splicing the feature map processed by the residual structure path with the feature maps processed by the local information extraction path and the spatial information extraction path.

[0073] For ordinary convolution, if the size of the input feature map is H×W×C1, that is, the height is H, the width is W, and the number of channels is C1. The size of the convolution kernel of ordinary convolution is 3×3. If the size of the output feature map is × ×C2, then its number of parameters and computational amount are calculated as follows:

[0074] ;

[0075] ;

[0076] For the SMK module, the size of the input feature is also H×W×C, and the formula for calculating the number of parameters is as follows:

[0077] ;

[0078] The formula for calculating the computational amount is as follows:

[0079] ;

[0080] If C1 = C2 = C, the number of parameters is , , and the number of parameters is reduced by approximately 48.5%, while the computational cost is , , and the computational cost is reduced by approximately 49%. In the scenario of steel surface defect detection where the features have different sizes, shapes, and texture characteristics, this embodiment introduces the SMK module, which can not only enhance the model detection efficiency, reduce the model complexity, but also make the network have stronger generalization ability and higher detection accuracy.

[0081] The SELA-S module of this embodiment is set between the original SPPF module and the C2PSA module in the backbone network, and includes a spatial attention mechanism path and a channel attention mechanism path for respectively processing the feature maps input to the SELA-S module. The processing results of the two paths are respectively multiplied by the input feature maps, and the multiplied results are added to the input feature maps and then used as the output.

[0082] Specifically, the spatial attention mechanism path in the SELA-S attention mechanism module of this embodiment includes two sets of spatially feature weight extraction modules connected in series. Each set of modules includes an average pooling layer, a feature extraction layer, group normalization, and a Sigmoid function connected in series. The average pooling layer of one set of modules is a vertical direction average pooling layer, and the other is a horizontal direction average pooling layer. The output of the first set of modules is multiplied by the feature map input to the SELA-S attention mechanism module and then input to the second set of modules. That is, the spatial attention mechanism includes two average pooling layers and feature extraction layers in series that are divided into two different vertical and horizontal directions, thereby obtaining spatial feature weights in two different directions.

[0083] The channel attention mechanism path includes an adaptive pooling layer and a convolutional layer connected in series in sequence, and finally passes through a sigmoid function to obtain channel feature weights.

[0084] Finally, the processing results of the two paths, including the spatial feature weights in the vertical and horizontal directions, and the channel feature weights obtained by the channel attention mechanism path are respectively multiplied by the original feature map, and each result is added to the original feature map as a residual branch and used as the output.

[0085] The specific implementation of the channel attention mechanism path is as follows. The feature map with an input size of H×W×C is subjected to global average pooling to generate a 1×1×C feature vector , so as to obtain a channel-level statistic with a global receptive field of channel C:

[0086] ;

[0087] Among them, represents the global average pooling operation, represents the row index (from 1 to H), represents the column index (from 1 to W).

[0088] Then, the channel - to - channel dependencies are learned through a single - layer convolution:

[0089] ;

[0090] Among them, s is the channel weight, represents the 1x1 convolution kernel parameter, represents the non - linear activation function Sigmoid.

[0091] For the other spatial attention path, first, for each channel c, average pooling is performed column - by - column in the width W direction to generate the horizontal - direction feature , and the formula is as follows:

[0092] ;

[0093] Among them, W represents the width of the feature map, H represents the height of the feature map, h represents the height - position index of the feature map, and its value range is , i represents the width - position index of the feature map, and its value range is , represents the feature value at the position (h, i) of channel c, that is, the value at the height h, width i, and channel c of the feature map.

[0094] Then, through a group of depth - wise separable convolutions including depth - wise convolution DWConv and point - wise convolution PWConv, features are further extracted, and then through group normalization Gn to enhance the position information, and finally through the Sigmoid activation function to obtain the spatial attention weight in the horizontal direction , and the formula is as follows:

[0095] ;

[0096] Then, the obtained weight is combined with the original feature map to obtain the feature map

[0097] ;

[0098] Then, based on the feature map , average pooling is performed column - by - column in the length H direction to generate the vertical - direction feature , and the formula is as follows:

[0099] ;

[0100] Among them, w represents the width position index of the feature map, and its value range is , j represents the height position index of the feature map, and its value range is , represents the feature value of channel c at position , that is, the value of the feature map at height j, width w, and channel c.

[0101] Then, it passes through a group of depthwise separable convolutions including depthwise convolution DWConv and pointwise convolution PWConv to further extract features, and then passes through group normalization Gn to enhance the position information. Finally, through the Sigmoid activation function to obtain the spatial attention weight in the vertical direction The formula is as follows:

[0102] ;

[0103] Finally, the channel attention mechanism path and the horizontal direction path are regarded as a residual structure, and added to the obtained output to obtain the output feature map :

[0104] ;

[0105] Therefore, in this embodiment, by introducing the SELA-S attention mechanism module, the sensitivity of the model to defect features can be improved, the attention of the model to the defect area can be enhanced, the interference of background information can be reduced, which helps the model to more accurately identify the defect area and improve the detection accuracy.

[0106] At the same time, the neck network of the detection model in this embodiment includes a twelve-layer module structure connected in series in sequence, where the first layer is the first upsampling module, the second layer is the first Fafusion module, the third layer is the first C3K2 module, the fourth layer is the second upsampling module, the fifth layer is the second Fafusion module, the sixth layer is the second C3K2 module, the seventh layer is the first Conv module, the eighth layer is the first Concat module, the ninth layer is the third C3K2 module, the tenth layer is the second Conv module, the eleventh layer is the second Concat module, and the twelfth layer is the fourth C3K2 module.

[0107] The input ends of the first Fafusion module are also respectively connected to the second C3K2 module, the third C3K2 module, and the fourth C3K2 module of the backbone network. The input ends of the second Fafusion module are also respectively connected to the first C3K2 module, the second C3K2 module, and the third C3K2 module of the backbone network. The output end of the second C3K2 module is also connected to the detection head. The input end of the first Concat module is also connected to the first C3K2 module of the neck network. The output end of the third C3K2 module is also connected to the detection head. The input end of the second Concat module is also connected to the C2PSA module of the backbone network.

[0108] The two feature fusion Fafusion modules introduced by the neck network of this embodiment. These two Fafusion modules are connected in series. The first Fafusion module is arranged after the first upsampling module of the neck network, and the output is processed by the first C3K2 module and the second upsampling module of the neck network and then used as the input of the second Fafusion module. Both Fafusion modules include four paths for processing feature maps. The processing results of the four paths are concatenated and the channels are adjusted to obtain the output. The four paths include: a downsampling path for large-size feature maps, including DepthwiseConv and PointwiseConv connected in series in sequence. A path for adjusting the number of channels of medium-size feature maps, including a Conv module for adjusting the number of channels of the feature maps. Two paths for small-size feature maps, both including an Unsample module. Finally, the output pictures of the four paths are concatenated and then passed through a Conv to adjust the channels to obtain the output feature map.

[0109] The reason why the multi-scale feature fusion pyramid network of the neck structure of this embodiment introduces the feature fusion module Fafusion on the basis of the existing neck structure is that the structural feature fusion of the existing Yolov11 relies on simple upsampling and splicing to integrate multi-level feature information, but this method is limited to continuous levels, thereby ignoring the fusion between other deep or shallow features. This local feature fusion method limits the global information sharing between features of different scales, resulting in limitations in the fusion of low-level fine-grained information and high-level semantic information. Moreover, the feature fusion between continuous levels often undergoes some other module processing, such as the C3k2 module and the CBS module in yolov11. This feature fusion method may cause a fault in the information transmission process, which weakens the key feature information held by the fused feature level. Therefore, in order to solve these problems, this embodiment provides a multi-scale feature fusion pyramid network, which fuses multi-scale feature levels. In the first feature fusion module Fafusion, the shallow features in the yolov11 backbone network, including the third C3K2 layer and the fourth C3K2 layer, and the deep features, including the sixth C3K2 layer, are fused with the upper module of Fafusion. In the second feature fusion module Fafusion, the shallow features in the yolov11 backbone network, including the fourth C3K2 layer and the sixth C3K2 layer, and the deep features, including the eighth C3K2 layer, are fused with the upper module of Fafusion. This enhances the network's information interaction capability at different scale feature levels and ensures global information sharing between different scale features of the network.

[0110] FaFusion will adjust the number of feature map channels and feature map size of the large-size feature map P3 from different levels through deep separable convolution, and the number of feature map channels of P4 through 1×1 standard convolution. The P5 and L1 layers with the same number of feature map channels but smaller sizes will be upsampled to adjust the feature map size. Finally, the processed feature maps of the four different levels will be concat, and the channels will be adjusted through 1×1 standard convolution to finally obtain the output feature map. By fusing multi-layer features of different feature levels, the FaFusion module can effectively integrate key features of different levels of the network, enhance the interactive fusion of low-level fine-grained information and high-level semantic information, and thus improve the expressiveness of multi-scale features. This makes the model more capable of identifying targets of different categories in complex scenarios, further improving detection accuracy and generalization performance.

[0111] For large feature maps ,have

[0112] ;

[0113] Among them, DWConv is depthwise convolution and PWConv is pointwise convolution

[0114] For medium-sized feature maps , there are

[0115] ;

[0116] For small-sized feature maps , there are

[0117] ;

[0118] Among them, Upsample is upsampling

[0119] For small-sized feature maps , there are

[0120] ;

[0121] Finally, and , , After performing the Concat operation and then adjusting the number of channels through Conv, the output feature map obtained is , then there are

[0122] .

[0123] After processing the surface image by the detection model of this embodiment, an image can be output that annotates the possible defect classification categories, bounding box coordinates, and the confidence level of the existence of defect targets.

[0124] When training the detection model of this embodiment, the GC10-DET dataset is used to construct the relevant dataset. This dataset contains 10 typical steel surface defects, namely punching (Pu), weld (Wl), crescent gap (Cg), water spot, oil spot (Os), wire spot (Ss), inclusion (In), rolling pit (Rp), crease (Cr), and waist crease (Wf). Since the original dataset also includes unlabeled data and some data with incorrect label names, this article first processes the dataset to remove unlabeled data, then corrects the labels of the data with incorrect labels, and then performs data augmentation methods including flipping, rotation, cropping, and brightness adjustment. Finally, the augmented dataset includes 4588 images.

[0125] Then, the dataset is divided into training set, test set, and validation set according to the ratio of 8:1:1. The training set is input into the untrained detection model for iterative training with 200 rounds. During the training process, the validation set is input to obtain the validation effect and adjust the model training strategy. Finally, the training effect is verified through the test set, and the performance of the model is evaluated to obtain the trained small target detection model for multi-scale feature interaction of steel surface defects.

[0126] During the iterative training process of this embodiment, multi-scale features are extracted by the backbone network, and then cross-layer feature interaction and fusion are realized by the neck network. Finally, the detection head outputs the classification category, bounding box coordinates, and target existence confidence as the results. Then, the results are compared with the label values, and backpropagation is performed to update the parameters of the detection model. At the same time, the validation set is regularly input into the detection model for verification to further adjust the parameters of the detection model according to the verification results. Finally, after verifying the training effect through the test set and evaluating the performance of the detection model, the parameters of the trained detection model are saved.

[0127] Among them, the detection head uses Focaler-PIoUv2, which is generated by combining the Focaler-IoU loss function and the PIoUv2 loss function, as the bounding box loss function during training. Since the essence of the bounding box loss function is to construct the bounding box regression loss function by quantifying the difference between the prediction parameters and the true annotation, continuously correcting the model, and improving the localization accuracy. The bounding box loss function used by Yolov11 is the CIoU loss function. CIoU considers more geometric factors on the basis of the traditional IoU. It constructs the bounding box regression loss function by considering the aspect ratio similarity, intersection over union, and center point normalized distance between the prediction box and the true box. However, the CIoU loss function still has limitations in complex scenarios. First, CIoU shows suboptimal performance when dealing with targets with extreme geometric shapes, especially when the target undergoes significant deformation or the aspect ratio distribution is discrete, which easily leads to positioning deviation. Second, due to the high complexity of the CIoU loss function, the gradient flow shows non-stationary characteristics during backpropagation, which brings difficulties to training. In addition, in the case of extremely small intersection over union, the positioning effect of CIoU will become worse. And PIoU is a bounding box regression loss function that enhances the bounding box regression by minimizing the Euclidean distance between the prediction box and the true box. This method provides a more intuitive similarity measure and is effective for both overlapping boxes and non-overlapping boxes. It reduces the problem of anchor box expansion and ensures accurate and stable bounding box regression. PIoU proposes a penalty factor P that adapts to the target size, which is defined as follows

[0128]

[0129] where and Indicates the distance between the predicted bounding box and the ground truth bounding box in the width direction. is the distance between the left edge of the predicted bounding box and the left edge of the ground truth bounding box, is the distance between the right edge of the predicted bounding box and the right edge of the ground truth bounding box. is the width of the ground truth bounding box, is the height of the ground truth bounding box. The result P is a penalty factor that can quantify the difference between the predicted bounding box and the ground truth bounding box and is dynamically adjusted according to the size of the ground truth bounding box, aiming to prevent the invalid expansion of the anchor box.

[0130] Then the definition of the first version of PIoU, PIoUv1, is as follows:

[0131]

[0132] where IoU is the intersection over union of the predicted bounding box and the ground truth bounding box, represents the natural constant.

[0133] And PIoUv2 used in this embodiment is obtained by further combining with a non - monotonic attention function controlled by a single hyperparameter on the basis of PIoUv1. The loss of PIoUv2 is obtained by adding an attention layer to the loss of PIoUv1, which enhances the focusing ability on medium - to - high - quality anchor boxes and improves the performance of the object detector. Therefore, the loss of PIoUv2 is expressed as:

[0134] .

[0135] where represents the hyperparameter that controls the behavior of the attention function. represents a parameter used to measure the quality of the anchor box, . At the same time, the non - monotonic attention function is:

[0136]

[0137] where represents the input value of the non - monotonic attention function.

[0138] Therefore, the loss of PIoUv2 is expressed as:

[0139]

[0140] If P is large (i.e., the difference between the predicted bounding box and the ground truth bounding box is large), the loss value L will also increase accordingly. Use q to measure the quality of the anchor box, It is an attention function that dynamically adjusts the weight of the loss function according to the value of the input x. λ is a hyperparameter that controls the behavior of the attention function. PIoUv2 improves the quality evaluation ability of the anchor boxes by introducing a penalty factor and a non-monotonic attention function. It quantifies the alignment degree between the anchor boxes and the ground truth boxes, enabling the model to pay more attention to the medium-quality anchor boxes, thereby optimizing the regression process. Therefore, PIoUv2 is a bounding box loss function that constructs the regression loss function by quantifying and minimizing the Euclidean distance between the predicted box and the ground truth box, and enhancing the bounding box regression with this. This method provides a more intuitive similarity measure and can guide the anchor boxes to regress along an effective path, not only solving the problem of slow convergence speed caused by the increase of traditional IoU anchor boxes but also ensuring more accurate and stable bounding box regression. Combining the two provides a more effective bounding box regression loss function for the multi-scale feature interaction steel surface defect small target detection model, enabling the model to focus on different regression samples in different detection tasks and ensuring more accurate and stable bounding box regression.

[0141] Focaler-IoU is a bounding box loss function designed to enable the model to focus on different regression samples in different detection tasks. Its formula is as follows:

[0142]

[0143] where is the value of the reconstructed Focaler-IoU, IoU is the original IoU value, and [d, u] ∈ [0, 1]. By adjusting the values of d and u, can focus on different regression samples. In this embodiment, the loss of Focaler-IoU is expressed as:

[0144] .

[0145] And represents the value of the reconstructed Focaler-IoU, and the expression is:

[0146] .

[0147] where represents the intersection over union of the predicted box and the ground truth box. and are two threshold parameters, and , ∈ [0, 1].

[0148] In summary, the loss of Focaler-PIoUv2 is expressed as:

[0149] 。

[0150] When specifically performing training, first divide the obtained dataset into a training set, a validation set, and a test set at a ratio of 8:1:1. Then construct a configuration yaml file corresponding to the dataset and fill in various parameters of the dataset, including the locations of the training set, validation set, and test set, the names of each category, and the number of categories. Then, by adjusting the parameter configuration file, the default.yaml file, this file can adjust the corresponding hyperparameters, such as the number of training iterations, the selection of the optimizer, etc. The experimental environment of this instance is as follows: the CPU is 15 vCPU Intel(R) Xeon(R) Platinum 8474C, the GPU is RTX 4090D (24GB) * 1, the Python version is 3.8 (ubuntu20.04), the deep learning framework is PyTorch 1.11.0, and the Cuda version is 11.3.

[0151] The set parameters are as follows: the initial learning rate is set to 0.01, the momentum decay parameter value is set to 0.937, the weight decay parameter value is set to 0.0005, the number of training epochs is set to 200, the Batchsize is set to 32, and end-to-end training is performed using the Stochastic Gradient Descent (SGD) optimizer. After preprocessing, the resolution of the input image is 640×640. Other parameters are set according to the default values. After setting, input the training set and the validation set into the multi-scale feature interaction small target detection model for steel surface defects for 200 rounds of iterative training. Images with a resolution of 640×640 are divided into each batch for dynamic data augmentation. In the first 190 rounds, four-image mosaic splicing is used. A batch of data after data augmentation will extract multi-scale features through the backbone network of the multi-scale feature interaction small target detection model for steel surface defects, and then achieve cross-layer feature interaction and fusion through the multi-scale feature fusion pyramid neck network. Finally, the decoupled detection head outputs the classification category, the bounding box coordinates, and the target existence confidence. And by comparing the results with the label values, backpropagation is performed to update the model parameters, and it can be verified according to the validation set. Adjust some hyperparameters according to the verification results to improve the network training effect. Finally, obtain the trained multi-scale feature interaction small target detection model for steel surface defects, and finally save the optimal weight file best.pt.

[0152] In specific implementation, a high-resolution linear array CCD camera is deployed on the continuous rolling production line of industrial steel to collect the surface images of steel plates in real time. After the images are subjected to pixel normalization (mean-variance standardization), Gaussian noise suppression, and non-uniform illumination compensation by the preprocessing module, they are input into the trained multi-scale feature interaction small target detection model for steel surface defects, and the images processed by the model are obtained, that is, the possible defect classification categories, bounding box coordinates, and the confidence of the existence of defect targets are marked on the steel surface images.

[0153] To verify the effectiveness of the multi-scale feature interaction small target detection model for steel surface defects proposed in the technical solution of this example, under the condition that the environment and experimental parameters are the same, the multi-scale feature interaction small target detection model (the technical solution of the present invention) is compared with the baseline model YOLOv11 in an experiment. The performance indicators include three indicators: mean average precision (mAP), inference speed (FPS), and number of parameters (Params).

[0154] The mean average precision (mAP) is used to measure the detection accuracy and comprehensiveness of different categories. The formula is as follows:

[0155] ;

[0156] ;

[0157] Among them, AP represents the accuracy of a single category, p(r) represents the precision corresponding to a specific recall rate r, mAP is the result obtained by integrating the area under the P-R curve, and AP i represents the precision of the i-th category, represents the mean of the accuracies of all categories, and n is the number of categories.

[0158] FPS represents the inference speed of the model, where represents the image preprocessing time, represents the image inference time, represents the image postprocessing time.

[0159] ;

[0160] The number of parameters (Params) refers to the parameters that need to be learned during the training process of the network model, representing the spatial complexity of the model. The larger the number of parameters, the larger the memory occupied by the model.

[0161] The experimental results are shown in Table 1. The mean average precision is the mean average precision of the model calculated when the intersection over union (IoU) threshold is set to 50%, that is, 0.5, and the result is written in the form of a percentage. Compared with the baseline model YOLOv11 in this embodiment, when the number of parameters slightly increases and the FPS slightly decreases, the average precision is increased by 4.7%, which improves the steel defect detection accuracy with almost no impact on the deployability and real-time performance of the model.

[0162] 。

[0163] Embodiment 2:

[0164] A detection system for small target detection of steel surface defects based on multi-scale feature interaction, including:

[0165] A preprocessing module: preprocesses the acquired steel surface defect images, that is, performs data augmentation operations on the data set using data augmentation methods including flipping, rotation, cropping, and brightness adjustment;

[0166] A model construction module: obtains a multi-scale feature interaction steel surface defect small target detection model by introducing the multi-scale feature fusion pyramid network designed in this paper, the feature fusion module Fafusion, the multi-scale feature extraction module SMK module, the attention mechanism module SELA-S, and the bounding box regression loss function Focaler-PIoUv2 into the YOLOv11 network;

[0167] The multi-scale feature interaction steel surface defect small target detection model includes a backbone network, a neck network, and a detection head connected in series in sequence. The backbone network introduces the multi-scale feature extraction module SMK module, and the output end of the C2PSA module is connected to the input end of the SELA-S attention mechanism module. The neck structure - the multi-scale feature fusion pyramid network introduces the feature fusion module Fafusion on the basis of the existing neck network, fuses the feature maps of different levels in the backbone network, and the bounding box regression loss function is Focaler-PIoUv2;

[0168] A training module: inputs the training set preprocessed by the preprocessing module into the multi-scale feature interaction steel surface defect small target detection model for training, inputs the validation set during the training process, adjusts the model training strategy according to the validation effect, and finally verifies the training effect through the test set to evaluate the performance of the model, so as to obtain a trained multi-scale feature interaction steel surface defect small target detection model;

[0169] Detection module: During the production of industrial steel, a camera device is used to obtain steel images, and the trained multi-scale feature interaction small target detection model for steel surface defects is input to obtain the category and location information of steel surface defects as well as the result confidence level.

[0170] The backbone network consists of a Conv module, a Conv module connected in series in sequence, and then connected to C3K2 modules, Conv modules, C3K2 modules, Conv modules, C3K2 modules, SMK module, C3K2 module, SPPF module, C2PSA module, and SELA-S attention mechanism module connected in series in sequence;

[0171] Among them, the Conv module includes a standard two-dimensional convolutional layer, a BatchNormalization layer, and an activation function SILU layer connected in series in sequence;

[0172] Among them, the C3K2 module includes a Conv module and a C3K module, and the C2PSA module includes a Conv module and a PSA module connected in series.

[0173] Among them, the SMK module, specifically as Figure 3 shown, includes three main paths, namely the residual structure path, the local information extraction path, and the spatial information extraction path. The local information path includes a Conv module and a channel attention mechanism module connected in series in sequence. The spatial information path includes two spatial information extraction paths with different scales. One path includes a 1×1Conv module and a 3×3Conv connected in series in sequence, and also includes a dilation=2 atrous convolution module for the x direction and a dilation=2 atrous convolution module for the y direction. The other path includes a 1×1Conv module and a 5×5Conv connected in series in sequence, and also includes a dilation=3 atrous convolution module for the x direction and a dilation=3 atrous convolution module for the y direction. Finally, the feature map processed by the residual structure path is concatenated with the feature maps processed by the local information extraction path and the spatial information extraction path to obtain the final output feature map.

[0174] Among them, the SELA-S attention mechanism module, specifically as Figure 4 shown, includes a spatial attention mechanism path and a channel attention mechanism path. Among them, the spatial attention mechanism includes two average pooling layers and a feature extraction layer divided into two different directions, vertical and horizontal, in series, so as to obtain spatial feature weights in two different directions. The channel attention mechanism is an adaptive pooling layer and a convolutional layer connected in series in sequence. Finally, through the sigmoid function, channel feature weights are obtained. Finally, each weight information is multiplied by the original feature map and added to the original feature map as the residual branch to obtain the output feature map.

[0175] Among them, the multi-scale feature fusion pyramid network of the neck structure adjusts the neck fusion route on the basis of the existing neck structure and introduces the feature fusion module Fafusion. Through the fusion of multi-scale feature levels, in the first feature fusion module Fafusion, the shallow features in the yolov11 backbone network, including the third C3K2 layer and the fourth C3K2 layer, the deep feature of the sixth C3K2 layer, and the upper module of the first feature fusion module Fafusion are fused. In the second feature fusion module Fafusion, the shallow features in the yolov11 backbone network, including the fourth C3K2 layer and the sixth C3K2 layer, the deep feature of the eighth C3K2 layer, and the upper module of the second feature fusion module Fafusion are fused.

[0176] Furthermore, the feature fusion module Fafusion is specifically as Figure 4 shown, including four paths. The first is the downsampling path for large-size feature maps, including DepthwiseConv and PointwiseConv connected in series in sequence. The second is the path for adjusting the number of channels of medium-size feature maps, including a Conv module for adjusting the number of channels of the feature map. The third and fourth paths respectively include two paths for small-size feature maps, and both paths include an Unsample module. Finally, the output images of the four paths are concatenated and then adjusted by Conv to obtain the output feature map.

[0177] Among them, Focaler-PIoUv2 is a bounding box loss function. Focaler-IoU is a bounding box loss function designed to enable the model to focus on different regression samples in different detection tasks, while PIoUv2 is a bounding box regression loss function constructed by quantifying and minimizing the Euclidean distance between the predicted box and the ground truth box, and enhancing the bounding box regression with this. This method provides a more intuitive similarity measure and can guide the anchor box to regress along the effective path, solving the problem of slow convergence speed caused by the increase of traditional IoU anchor boxes, and ensuring accurate and stable bounding box regression. The combination of the two is the Focaler-PIoUv2 bounding box loss function, which provides a more effective bounding box regression loss function for the multi-scale feature interaction steel surface defect small target detection model, enabling the model to focus on different regression samples in different detection tasks and ensuring more accurate and stable bounding box regression.

[0178] It should be understood that for the specific implementation processes of each module, please refer to the above method content, which will not be elaborated herein. Moreover, the above division of functional modules is only for illustrative purposes. In some embodiments, some functional modules can be combined, some can be split, and each functional module can be implemented in software, hardware, or a combination of both. The software and hardware devices include, but are not limited to, general computer devices, programmable gate arrays, digital signal processors, microprocessors, and their corresponding programming or burning software.

[0179] Embodiment 3:

[0180] A computer-readable storage medium stores a computer program, and the computer program is called by a processor to implement:

[0181] The steps of the above steel surface defect detection method.

[0182] For the specific implementation processes of each step, please refer to the elaboration of the foregoing method.

[0183] It should be understood that in the embodiments of the present invention, the so-called processor may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A part of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0184] Embodiment 4:

[0185] The present invention provides a computer-readable storage medium that stores a computer program, and the computer program is called by a processor to implement: the steps of the above steel surface defect detection method.

[0186] For the specific implementation processes of each step, please refer to the elaboration of the foregoing method.

[0187] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the software and hardware device in any of the foregoing embodiments, such as the hard disk or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk equipped on the controller, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the readable storage medium can also include both the internal storage unit of the controller and the external storage device. The readable storage medium is used to store computer programs and other programs and data required by the controller. The readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0188] Based on such understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present invention. The foregoing readable storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0189] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code. The present application is a device that generates, according to the flowchart of the method, device (system), and computer program product according to the embodiments of the present application and the instructions executed by the processor, for realizing the functions specified in one process or multiple processes of the flowchart and / or one block or multiple blocks of the block diagram. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in one process or multiple processes of the flowchart and / or one block or multiple blocks of the block diagram. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one process or multiple processes of the flowchart and / or one block or multiple blocks of the block diagram.

[0190] It should be emphasized that the examples of the present invention are illustrative rather than restrictive. Therefore, the present invention is not limited to the examples in the specific embodiments. Any other embodiments obtained by those skilled in the art according to the technical solutions of the present invention, without departing from the spirit and scope of the present invention, whether modified or replaced, also belong to the protection scope of the present invention.

Claims

1. A small target detection method for surface defects based on multi-scale feature interaction, characterized in that It includes the following steps: Step 1: Input the surface image of the product to be detected collected in real time into the detection model; Step 2: After the detection model processes the surface image, output an image annotating the possible defect classification categories, bounding box coordinates, and the confidence level of the existence of defect targets; Among them, the detection model is a model based on the YOLOv11 network as the basic structure, including a backbone network, a neck network, and a detection head connected in series in sequence; The multi-scale feature extraction SMK module and the attention mechanism SELA-S module are introduced into the backbone network; two feature fusion Fafusion modules are introduced into the neck network; The SMK module is set between the original third and fourth C3K2 modules of the backbone network, and includes a residual structure path, a local information extraction path, and a spatial information extraction path for respectively processing the feature maps input into the SMK module. The processing results of the three paths are spliced and used as the output; The SELA-S module is set between the original SPPF module and the C2PSA module of the backbone network, and includes a spatial attention mechanism path and a channel attention mechanism path for respectively processing the feature maps input into the SELA-S module. The processing results of the two paths are respectively multiplied by the input feature maps, and the multiplied results are added to the input feature maps and then used as the output; The two Fafusion modules are connected in series. Among them, the first Fafusion module is set after the first upsampling module of the neck network, and the output is processed by the first C3K2 module and the second upsampling module of the neck network and then used as the input of the second Fafusion module; both Fafusion modules include four paths for processing feature maps, and the processing results of the four paths are spliced and the channels are adjusted and then used as the output.

2. The method according to claim 1, wherein In Step 1, the detection model is obtained through the following steps of training: Step 101: Obtain the surface images of products with surface defects, annotate them as a data set, and then perform data augmentation on the images in the data set to expand the data set; Step 102: Divide the expanded data set into a training set, a validation set, and a test set according to a preset ratio. Then input the training set into the detection model for iterative training, and regularly input the validation set into the detection model during the iterative training process, so as to adjust the model training strategy according to the validation effect; finally, after the iterative training is completed, verify the training effect and evaluate the performance of the detection model by inputting the test set.

3. The method according to claim 2, wherein In Step 101, the data augmentation is to perform processing including flipping, rotating, cropping, and brightness adjustment on the images, so as to generate new processed images based on the original images to expand the data set.

4. The method according to claim 2, wherein In step 102, during the iterative training process, multi-scale features are extracted by the backbone network, then cross-layer feature interaction and fusion are achieved by the neck network, and finally the classification category, bounding box coordinates, and object existence confidence are output by the detection head as the results; then the results are compared with the label values, and backpropagation is performed to update the parameters of the detection model; at the same time, the validation set is regularly input into the detection model for validation to further adjust the parameters of the detection model according to the validation results; finally, after verifying the training effect through the test set and evaluating the performance of the detection model, the parameters of the trained detection model are saved.

5. The method according to claim 2, characterized in that In step 102, the detection head uses Focaler-PIoUv2, which is generated by combining the Focaler-IoU loss function and the PIoUv2 loss function, as the bounding box loss function during the training process; Among them, the loss of Focaler-IoU is expressed as: ； while represents the reconstructed Focaler-IoU value, and the expression is: ； where represents the intersection over union of the predicted bounding box and the ground truth bounding box; and are two threshold parameters, and , ∈ [0, 1]; PIoUv2 is obtained by combining the PIoUv1 loss function and a non - monotonic attention function controlled by a single hyperparameter The expression of PIoUv1 is as follows: ； represents the natural constant; represents the penalty factor, and the expression is: ； Among them is the distance between the left edge of the predicted box and the left edge of the ground truth box, is the distance between the right edge of the predicted box and the right edge of the ground truth box, is the distance between the upper edge of the predicted box and the upper edge of the ground truth box, is the distance between the lower edge of the predicted box and the lower edge of the ground truth box, is the width of the ground truth box, is the height of the ground truth box; Non-monotonic attention function The expression is as follows: ； wherein represents the input value of the non-monotonic attention function; Therefore, the loss of PIoUv2 is expressed as: ； wherein represents a hyperparameter for controlling the behavior of the attention function; represents a parameter for measuring the quality of the anchor box, ; In summary, the loss of Focaler-PIoUv2 The expression is: 。 6. The method according to claim 1, wherein The backbone network of the detection model includes a twelve-layer module structure connected in series in sequence, where the first layer is the first Conv module, the second layer is the second Conv module, the third layer is the first C3K2 module, the fourth layer is the third Conv module, the fifth layer is the second C3K2 module, the sixth layer is the fourth Conv module, the seventh layer is the third C3K2 module, the eighth layer is the SMK module, the ninth layer is the fourth C3K2 module, the tenth layer is the SPPF module, the eleventh layer is the SELA-S attention mechanism module, and the twelfth layer is the C2PSA module.

7. The method according to claim 6, characterized in that, The SMK module includes: A residual structure path, including a 1×1 Conv module; A local information extraction path, including a Conv module and a channel attention mechanism module connected in series in sequence; A spatial information extraction path, including two sub-paths with different scales, one of which includes a 1×1 Conv module, a 3×3 Conv module, a dilated convolution module with a dilation of 2 for the x direction, and a dilated convolution module with a dilation of 2 for the y direction connected in series in sequence, and the other includes a 1×1 Conv module, a 5×5 Conv module, a dilated convolution module with a dilation of 3 for the x direction, and a dilated convolution module with a dilation of 3 for the y direction connected in series in sequence; A Concat module for splicing the residual structure path, the local information extraction path, and the spatial information extraction path.

8. The method according to claim 6, wherein The SELA-S attention mechanism module includes: A spatial attention mechanism path, including two sets of spatially feature weight extraction modules connected in series, each set of modules includes an average pooling layer, a feature extraction layer, group normalization, and a Sigmoid function connected in series, where the average pooling layer of one set of modules is a vertical direction average pooling layer, and the other is a horizontal direction average pooling layer; the output of the first set of modules is multiplied by the feature map input to the SELA-S attention mechanism module and then input to the second set of modules to obtain spatially feature weights in two different directions; The channel attention mechanism path includes an adaptive pooling layer and a convolutional layer connected in series in sequence, and finally passes through a sigmoid function to obtain channel feature weights; Then, the processing results of the two paths, including the spatial feature weights in the vertical and horizontal directions and the channel feature weights obtained by the channel attention mechanism path, are multiplied by the original feature map respectively. Finally, each result is added to the original feature map serving as the residual branch to obtain the output.

9. The method according to claim 6, wherein The neck network of the detection model includes twelve sequentially connected module structures. The first layer is the first upsampling module, the second layer is the first Fafusion module, the third layer is the first C3K2 module, the fourth layer is the second upsampling module, the fifth layer is the second Fafusion module, the sixth layer is the second C3K2 module, the seventh layer is the first Conv module, the eighth layer is the first Concat module, the ninth layer is the third C3K2 module, the tenth layer is the second Conv module, the eleventh layer is the second Concat module, and the twelfth layer is the fourth C3K2 module. The input end of the first Fafusion module is also respectively connected to the second C3K2 module, the third C3K2 module, and the fourth C3K2 module of the backbone network; the input end of the second Fafusion module is also respectively connected to the first C3K2 module, the second C3K2 module, and the third C3K2 module of the backbone network; the output end of the second C3K2 module is also connected to the detection head; the input end of the first Concat module is also connected to the first C3K2 module of the neck network; the output end of the third C3K2 module is also connected to the detection head; the input end of the second Concat module is also connected to the C2PSA module of the backbone network.

10. The method according to claim 1, wherein The Fafusion module in the neck network has four paths, including: a downsampling path for large-size feature maps, which includes a sequentially connected DepthwiseConv and PointwiseConv; a path for adjusting the number of channels of medium-size feature maps, which includes a Conv module for adjusting the number of channels of the feature map; two paths for small-size feature maps, both of which include an Unsample module. Finally, the output images of the four paths are concatenated, and then the channels are adjusted through Conv to obtain the output feature map.

Images