Dual-mode target detection algorithm and system based on steel surface defects

Abstract

The invention relates to the technical field of steel surface detection, in particular to a bimodal target detection algorithm and system based on steel surface defects, and the algorithm comprises the steps: collecting steel surface data, and carrying out the processing to obtain a laser reflection intensity image representing two-dimensional texture information and a surface depth image representing three-dimensional depth information; and constructing a bimodal feature extraction and fusion network model: introducing deformable convolution into a YOLOv8 backbone network, constructing a parallel feature extraction strategy to realize feature fusion, fusing bimodal image features after feature extraction, and meanwhile, using an RGB image and a depth image as input of the network, so as to obtain a bimodal feature extraction and fusion network model. Optimizing a model training process by adopting a PloU loss function and a CloU loss function which are used for performing exponentiation on the IoU, and identifying an output result; according to the method, the detection rate of steel surface defects is greatly improved, the yield of products can be effectively guaranteed, and meanwhile the influence of external environmental factors on an algorithm is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of steel surface detection, and in particular to a dual-modal object detection algorithm and system based on steel surface defects. Background Art

[0002] As an important basic industry of the national economy, the quality control of steel products occupies an important position. A perfect quality inspection system can effectively ensure the yield rate of products. However, due to the uncertainty of the production environment and the constraints of the production process, some defects such as pitting, patches, scratches, etc. often occur on the steel surface, which greatly affects the performance of steel products. Affected by steel surface fouling, oxidation and rust, water stains and oil stains, etc., the background of the collected steel surface image information is very complex. It is difficult to distinguish pseudo-defects from real defects only by using the color and texture information provided by RGB images or grayscale images. In addition, depth information, as an important index for evaluating the damage degree of the steel surface, has great research value. Therefore, it is an effective solution to perform defect detection by complementary fusion of the dominant features of two different modal images. Summary of the Invention

[0003] The present invention provides a dual-modal object detection algorithm and system based on steel surface defects, which greatly improves the detection rate of steel surface defects, can effectively ensure the yield rate of products, and at the same time reduces the influence of external environmental factors on the algorithm.

[0004] To achieve the above object, the present invention is implemented by the following technical solutions:

[0005] A dual-modal object detection algorithm based on steel surface defects, comprising the following steps:

[0006] S1. Collect steel surface data, and process to obtain a laser reflection intensity image representing two-dimensional texture information and a surface depth image representing three-dimensional depth information;

[0007] S2. Build a dual-modal feature extraction and fusion network model: Introduce deformable convolution into the YOLOv8 backbone network, construct a parallel feature extraction strategy to achieve feature fusion, fuse the dual-modal image features after feature extraction, and use RGB images and depth images as the network inputs at the same time;

[0008] S3. Optimize the loss function: Use the PloU loss function and the CloU loss function that perform power operations on IoU to optimize the model training process;

[0009] S4. Identify the output result.

[0010] Further, the specific steps of step S1 are as follows: The light knife plane of the linear laser is vertically incident on the steel surface to obtain a laser cross-sectional image of the steel component. Coordinate transformation converts the image pixel coordinates into spatial physical coordinates, and intensity mapping is performed on the physical coordinate positions to obtain an intensity map representing two-dimensional features. At the same time, depth mapping is performed to obtain a depth map representing three-dimensional features. The pixels between the generated intensity map and depth map are in one-to-one correspondence.

[0011] Further, the specific steps of step S2 include the following steps:

[0012] S2.1. Improve the backbone network of YOLOv8: Apply deformable convolution to the backbone network of YOLOv8, which is called DeformBackbone. The network input size is set to 640×640. DeformBackbone has 5 convolutional layers Conv layers, 4 C2f, and 1 SPPF. The types of convolutional layers include conventional convolutional layer Conv and deformable convolutional layer Deform-Conv. The deformable convolutional layer Deform-Conv is set between every 2 C2Fs. The Bottleneck network in the C2F convolutional layer contains a deformable residual block, denoted as DeformBottleneck;

[0013] The Bottleneck network includes 2 3×3 conventional convolutional layers and 1 shortcut layer. When the shortcut is True, DeformBottleneck replaces the 2 3×3 conventional convolutional layers with deformable convolution. At the same time, batch normalization and activation functions are added after the deformable convolutional layer;

[0014] S2.2. Dual-modal feature fusion module;

[0015] The first stage:

[0016] Concatenate the RGB feature map F RGB extracted by the two-stream backbone network and the depth feature map F Depth to obtain a dual-modal feature map F RGB-D , as shown in Equation (1),

[0017] F RGB-D = Concat(F RGB ; F Depth ) (1)

[0018] Through the depthwise separable convolution module DSConv 3*3 perform on F RGB-DPerform a convolution operation to integrate RGB information and depth information to construct a correlation representation. The parallel patch-aware attention module PPA captures feature information at different scales and levels through a multi-branch feature extraction and fusion module and an adaptive attention mechanism module;

[0019] By dividing the input features into local patches of p×p, local context information is captured, and the features within each patch are averaged. The parallel patch-aware attention module filters important features through a multi-layer perceptron MLP and the activation function Softmax, and fuses the local context using a residual weighting method to finally generate local features F local and global features F glocal , the cascaded convolution consists of three 3×3 convolutional layers in serial convolution, gradually fusing local and context information to obtain the serial convolution output F conv , fuse the features of the above three branches to obtain an integrated feature representation As shown in formula (2):

[0020]

[0021] After multi-branch feature extraction, an adaptive attention mechanism is used for enhancement in the adaptive channel and spatial dimensions. It is successively processed by a one-dimensional channel attention map M C and a two-dimensional spatial attention map M S , and the specific formula is as follows:

[0022]

[0023] where represents element-wise multiplication, F C and F S represent the features after channel and spatial selection respectively, δ, and represent the rectified linear unit, batch normalization, and the regularization function dropout respectively, and F' is the final output of PPA;

[0024] The second stage: Construct a residual connection structure Res to filter redundant information and highlight key information, and construct the feature representation F” of the defect in the three-dimensional space, as shown in formula (6).

[0025] F” = F' + Res(F') (6)

[0026] Furthermore, the multi-branch fusion in the multi-branch feature extraction and fusion module includes patch awareness and cascaded convolution. The patch parameter p in patch awareness is set to 2 or 4. When P is set to 2, it represents the local branch, and when set to 4, it represents the global branch. Different branches use patches of different sizes.

[0027] Further, the specific steps of step S3 include performing a power operation on IoU, and the PloU loss function is expressed as:

[0028] L PIoU = 1 - IoU p (p > 1) (7)

[0029] where p is a hyperparameter used to balance the overlap degree between the predicted bounding box and the ground truth bounding box;

[0030] At different stages of training, dynamically adjust the weights of PIoU and CloU according to the IoU values of the predicted bounding box and the ground truth bounding box, and combine the advantages of both; after combination, the loss function L CPIoU as shown in (8)

[0031] L CPIoU = βL PIoU + (1 - β)L CIoU (8)

[0032] In the formula, L CIoU is the CIoU loss function, β is the dynamic weight, and β is adjusted according to the value of IoU through the Sigmoid function;

[0033]

[0034] where k is a hyperparameter controlling the weight adjustment speed, and t is a hyperparameter controlling the weight adjustment threshold.

[0035] Further, the specific steps of step S4 include, based on the collected data, customarily dividing the data set into multiple typical defect types, using the mean average precision mAP and the number of frames detected per second FPS as evaluation metrics, training the model, and inputting the image data to be detected into the model described in step S2 that has been trained for detection to obtain the detection result.

[0036] A dual-modal object detection system based on steel surface defects, which is a line-structured light three-dimensional measurement system, includes a linear laser, a lens, and a 3D camera;

[0037] The linear laser is used to vertically irradiate the light knife onto the steel surface;

[0038] The 3D camera is used to obtain the laser cross-sectional image of the steel component;

[0039] The lens is used to focus and project the light knife emitted by the linear laser onto the rail surface, and image the modulated light strip onto the sensor of the 3D camera.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] Aiming at the problem that the current RGB images of the steel surface lack the key depth information of the third dimension and the false alarm rate of the surface defect detection results is high, a dual-modal object detection algorithm and system based on steel surface defects are proposed. By constructing a dual-branch feature extraction network for steel surface defects, the depth image and RGB image of the steel surface are respectively subjected to feature extraction. In order to reduce the interference between the features of low-quality images and make full use of the complementary information of the dual modality, a dual-modal feature fusion module is used in the intermediate fusion stage to improve the integration ability of multi-modal information. In addition, by introducing the PIoU2 loss function, the training process of the model is optimized. The experimental results show that this method can significantly reduce the detection false alarm while maintaining a detection rate of more than 95%. Compared with the common deep learning detection models in current defect detection, while ensuring a high detection speed, the mean average precision mAP has increased by more than 5%, and it has stronger robustness.

[0042] In the present invention, the advantageous features of two different modality images are complementarily fused for defect detection, which can significantly reduce the detection false alarm while maintaining a high detection rate, greatly improving the detection rate of steel surface defects, effectively ensuring the yield rate of products, and at the same time reducing the influence of external environmental factors on the algorithm. On this basis, work will also be carried out to optimize the model parameter quantity to improve the detection rate, so that the model can further meet the actual deployment and application requirements of industrial equipment, thus bringing more extensive application value in actual production scenarios and having stronger robustness, with broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the method described in the present invention.

[0044] Figure 2 is a schematic diagram of multi-source data imaging described in the present invention.

[0045] Figure 3 is a schematic diagram of the structure of the dual-modal object detection algorithm based on steel surface defects described in the present invention.

[0046] Figure 4 is a schematic diagram of the structure of the dual-modal feature fusion module described in the present invention.

[0047] Figure 5 is a schematic diagram of the experimental visualization results described in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0048] The following further describes the specific embodiments of the present invention with reference to the drawings:

[0049] See Figure 1 , which is a flowchart of the method described in the present invention. A dual-modal object detection algorithm based on steel surface defects in the present invention includes the following steps:

[0050] S1. Collect steel surface data through a dual-modal structured light sensor. After processing, a laser reflection intensity image representing two-dimensional texture information and a surface depth image representing three-dimensional depth information are obtained. The dual-modal structured light sensor is a line-structured light three-dimensional measurement system, which consists of a line laser, a lens, and a camera. The laser line is incident on the surface of the measured object and is modulated into a light strip reflecting the contour information of the measured object. The camera captures the light strip to obtain the laser cross-section image of the measured object. The center pixel coordinates of the light strip are extracted from this image. Combining the pixel coordinates of the light strip center and the system calibration parameters, the actual contour of the measured object can be calculated. Based on this principle, a dual-modal target detection system based on steel surface defects of the present invention is shown in Figure 2 , that is, a multi-source data acquisition system, which is also a line-structured light three-dimensional measurement system, consisting of a 3D camera, a lens, and a linear laser. The light knife plane of the linear laser is vertically incident on the steel surface. The 3D camera obtains the laser cross-section image of components such as steel. The image pixel coordinates are converted into spatial physical coordinates through camera coordinate transformation, and intensity mapping is performed on the physical coordinate positions to obtain an intensity map representing two-dimensional features. At the same time, depth mapping is performed to obtain a depth map representing three-dimensional features. The pixels between the generated intensity map and depth map are in one-to-one correspondence.

[0051] S2. Build a dual-modal feature extraction and fusion network structure;

[0052] Based on the improved YOLOv8, a parallel feature extraction strategy is constructed to achieve feature fusion. After feature extraction, the dual-modal image features are fused. This fusion strategy can ensure that the backbone network can retain the independent key features of each modality;

[0053] S2.1. Improve the backbone network of YOLOv8;

[0054] The backbone network is crucial for achieving the best performance as a feature extractor. Deformable convolution can improve the generalization ability of the model to irregular shape defect models and has been proven to be an effective method to improve the performance of deep networks. Since conventional convolution kernels have poor adaptability and generalization ability to unknown changes, for objects with more complex shapes, normal convolution may not work well. Deformable convolution introduces a learnable offset that can better adjust its shape adaptation ability according to the actual situation. There are irregular interferences in steel surface defects, and the defect shapes are randomly irregular, which conforms to the characteristic description of deformable convolution;

[0055] The innovation of the present invention is to introduce deformable convolution into the YOLOv8 backbone network to enhance the performance of the backbone network. Specifically, deformable convolution is applied to the YOLOv8 backbone network, which is called DeformBackbone. The entire backbone network structure is as Figure 3As shown, the network input size is set to 640×640. DeformBackbone mainly has 5 convolutional layers Convlayers, 4 C2fs, and 1 SPPF. Among them, the types of convolutional layers include the conventional convolutional layer Conv and the deformable convolutional layer Deform-Conv. The deformable convolutional layer Deform-Conv is set between every two C2Fs. The Bottleneck network in the C2F convolutional layer contains a deformable residual block, denoted as DeformBottleneck. The detailed parameters of each convolutional layer and residual block are shown in Table 1:

[0056] Table 1 Detailed Parameters of DeformBackbone

[0057]

[0058]

[0059] The conventional Bottleneck network includes 2 3×3 conventional convolutional layers and 1 shortcut layer. When the shortcut is True, DeformBottleneck replaces these 2 3×3 conventional convolutional layers with deformable convolutional layers. At the same time, batch normalization and activation functions are added after the deformable convolutional layers. The shortcut layer is similar to the shortcut layer in the ResNet network, which can add each input feature map element by element as the output, facilitating the learning of features at different scales, reducing the training difficulty, and improving the model accuracy;

[0060] In addition, both RGB images and depth images are used as the network input at the same time. Therefore, the present invention adopts a parallel feature extraction strategy, that is, two backbone networks are created to extract the features of the two inputs respectively. Since the deformable convolution is introduced to solve the problems of object size, pose, and deformation, for the traditional RGB image of visible light intensity reflection, it can effectively improve the recognition rate of the depth network. The steel surface depth image contains rich depth distance information. Through this feature, noise interferences such as water stains and dirt on the steel surface can be effectively distinguished. However, the depth image has a weaker ability to express texture details and surface deformation features than the RGB image, and the effect of applying the deformable convolution network to it is not obvious. Moreover, since the deformable convolution makes the model more sensitive to noise while improving the network recognition ability, and there is a large amount of environmental interference noise in the real steel surface, this will inevitably affect the detection effect of the depth image. Therefore, the basic idea of the dual-modal fusion in the present invention is to use the deformable convolution for the RGB image to improve the detection ability of the network, while the depth image retains the conventional convolution kernel to suppress noise interference and reduce the model false alarm;

[0061] S2.2. Dual-Modal Feature Fusion Module;

[0062] In the RGB image, the complex background and low-contrast defects on the steel surface make it difficult to distinguish defects from the background. When the depth of the depth image is small, the defects are easily submerged in the complex background. According to the characteristics of RGB images and depth images themselves, common feature fusion methods, such as addition and concatenation, often ignore the influence of these low-quality images and the differences between different modality features, and cannot make full use of the complementary characteristics of bimodal information, resulting in one modality feature suppressing the representation of the other modality feature, or mutually supplementing misleading feature information, which instead has a negative effect;

[0063] Based on the above two points, to avoid the mutual interference of bimodal information and make full use of the complementary characteristics of bimodal information, a parallel patch-aware attention (PPA) module is designed and introduced. The module structure is as Figure 4 shown; in order to reduce the number of parameters of the module and avoid model overfitting, depthwise separable convolutions are used instead of traditional convolutions;

[0064] Feature fusion is divided into two stages. In the first stage, the RGB feature map F RGB extracted by the two-stream backbone network and the depth feature map F Depth are concatenated to obtain the bimodal feature map F RGB-D , as shown in Equation (10):

[0065] F RGB-D = Concat(F RGB ; F Depth ) (10)

[0066] The depthwise separable convolution module DSConv 3*3 performs a convolution operation on F RGB-D to integrate RGB information and depth information to construct a correlation representation. Then, the PPA captures feature information at different scales and levels through its unique multi-branch feature extraction and fusion module and adaptive attention mechanism module, enhancing the object detection ability in complex environments;

[0067] The multi-branch fusion includes patch awareness and cascaded convolutions. The patch parameter p in patch awareness is set to 2 or 4. When P is set to 2, it represents the local branch, and when set to 4, it represents the global branch. Different branches use patches of different sizes, which can effectively model multi-scale features and enhance the representational ability of complex features. Specifically, by dividing the input features into local patches of p×p, local context information is captured, and the features within each patch are averaged to generate a concise feature representation. On this basis, the module screens important features through a multi-layer perceptron (MLP) and the activation function Softmax, and uses the residual weighting method to fuse the local context, finally generating the local feature F local and the global feature F glocal, the cascaded convolution consists of three 3×3 convolutional layers in serial convolution, gradually fusing local and context information to obtain the serial convolution output F conv , fuse the features of the above three branches to obtain the comprehensive feature representation As shown in formula (11):

[0068]

[0069] After multi-branch feature extraction, an adaptive attention mechanism is used for enhancement in the adaptive channel and spatial dimensions, which is successively processed by a one-dimensional channel attention map M C and a two-dimensional spatial attention map M S The specific formula is as follows:

[0070]

[0071]

[0072] where denotes element-wise multiplication, F C and F S represent the features after channel and spatial selection respectively, δ, and represent the rectified linear unit, batch normalization, and the regularization function dropout respectively, and F' is the final output of PPA;

[0073] In the second stage, a residual connection structure Res is constructed to filter redundant information and highlight key information, constructing the feature representation F” of the defect in three-dimensional space, as shown in formula (15).

[0074] F” = F' + Res(F') (15)

[0075] S3, loss function optimization;

[0076] The original YOLOv8 model uses the CloU loss function, which is an improved IoU loss function. It considers information such as the distance between the center points of the predicted box and the target box, and the consistency of the aspect ratio. However, in the initial stage of regression, in order to overlap with the target box, the anchor box increases, which will lead to an inefficient regression path. And PIoU is the power operation on the traditional IoU, and the loss function L PIoU As shown in formula (16):

[0077] L PIoU = 1 - IoU p (p > 1) (16)

[0078] where p is a hyperparameter used to balance the overlap degree between the predicted box and the true box;

[0079] In the initial stage of training the object detection model, when the IoU value between the predicted bounding box and the ground truth bounding box is lower than 0.3, PIoU will increase the gradient of the loss function, enabling the model to more quickly adjust the approximate position of the predicted bounding box and update the parameters in the correct direction. Therefore, at different stages of training, we dynamically adjust the weights of PIoU and CloU according to the IoU value between the predicted bounding box and the ground truth bounding box, thereby combining the advantages of both;

[0080] The combined loss function L CPIoU as shown in (17)

[0081] L CPIoU = βL PIoU + (1 - β)L CIoU (17)

[0082] In the formula, L CIoU is the CIoU loss function, β is the dynamic weight, where β is adjusted according to the IoU value through the Sigmoid function;

[0083]

[0084] where k is a hyperparameter that controls the weight adjustment speed, and t is a hyperparameter that controls the weight adjustment threshold;

[0085] At the beginning stage of training, when the IoU value between the predicted bounding box and the ground truth bounding box is less than 0.3, by increasing the weight of PIoU, the model amplifies the enhanced gradient of PIoU to 3 times the IoU loss, thereby increasing the update speed of the predicted bounding box position parameters by 40% - 50%, enabling the predicted bounding box to quickly approach the ground truth bounding box. As the training progresses, when the IoU value between the predicted bounding box and the ground truth bounding box increases above 0.5, reduce the weight dependence on PIoU and rely on the CloU loss. The factors such as distance and aspect ratio comprehensively considered by CloU can help the model make more refined adjustments to the predicted bounding box and improve the detection accuracy.

[0086] S4. Identify the output result;

[0087] Based on the collected data, the dataset is custom - divided into multiple typical defect types, and the mean average precision mAP and frames per second FPS are used as the main evaluation metrics to train the model. The image data to be detected is input into the model described in step S2 that has been trained for detection to obtain the detection result.

[0088] This implementation case was carried out under the Pytorch 2.1.0+cu121 deep learning framework. The experimental environment was under the Windows 10 operating system, with the GPU being NVIDIA GeForce RTX 3080 (24GB), the processor being 12th Gen Inter(R)Corce(TM)i5-12600kF, the memory being 32G, and the Python version being 3.8. In this experiment, the industrial RGB-D dataset NEURSDDS-AUG was used as the experimental dataset. In terms of hyperparameter settings, the image resolution was dynamically adjusted to 640*640 pixel size before the image was input into the network. In addition, the initial learning rate (learning rate, lr) and the number of iterations (epoch) were set to 2×10-2 and 300 respectively. Among them, the first 50 iterations were frozen training, and only the parameters other than the two-stream backbone network were trained. The batch size was 64. After that, the backbone network was unfrozen for training, and the batch size became 32. The learning rate was adjusted using a strategy that combined the cosine annealing algorithm (cosine annealing, Cos) and the adaptive moment estimation (Adaptive Moment Estimation, Adam) optimization algorithm to better adjust the learning rate to accelerate the convergence of the model. Among them, the momentum Momentum was set to 0.937. The designed bimodal feature extraction and fusion network model was trained, and the trained model was compared with other commonly used models in the field of steel surface defect detection based on a single modality. The experimental results are shown in Table 2 below:

[0089] Table 2 Comparative Experiments

[0090] Network model mAP@50 / % FPS YOLOv8n 95.4 117 YOLOv8s 96.6 82 YOLOv5s 96.0 77 The present invention 97.2 93

[0091] Compared with the original model YOLOv8, the mAP@50 of the present invention has increased by 1.8%. Compared with YOLOv8s, which has the best comprehensive performance in the steel surface defect detection task, the mAP@50 of the present invention has increased by 0.6, proving that the model effectively improves the detection effect while ensuring high detection efficiency. Figure 5 This is the recognition effect diagram of the model proposed by the present invention on the computer.

[0092] The above embodiments are implemented on the premise of the technical solution of the present invention, and the detailed implementation methods and specific operation processes are given. However, the protection scope of the present invention is not limited to the above embodiments. The methods used in the above embodiments are all conventional methods unless otherwise specified.

Claims

1. A dual-modal target detection algorithm based on steel surface defects, characterized in that: The steps include: S1, collecting steel surface data, processing to obtain a laser reflection intensity image representing two-dimensional texture information and a surface depth image representing three-dimensional depth information; S2. Build a dual-modal feature extraction and fusion network model: Introduce deformable convolution into the YOLOv8 backbone network, build a parallel feature extraction strategy to achieve feature fusion, fuse the dual-modal image features after feature extraction, and use RGB images and depth images as network inputs; S3, loss function optimization: the PloU loss function and CloU loss function that perform power operation on IoU are used to optimize the model training process; S4. Identify the output result.

2. The dual-modal target detection algorithm based on steel surface defects according to claim 1 is characterized in that: Specifically, step S1 comprises: the light knife plane of the linear laser is incident vertically onto the steel surface, a laser cross-sectional image of the steel component is obtained, coordinate transformation converts the image pixel coordinates into spatial physical coordinates, and intensity mapping is performed on the physical coordinate positions to obtain an intensity map representing two-dimensional features, while depth mapping is performed to obtain a depth map representing three-dimensional features, and the pixels between the generated intensity map and depth map are one-to-one corresponding.

3. The dual-modal target detection algorithm based on steel surface defects according to claim 1, characterized in that: The step S2 specifically includes the following steps: S2.

1. Improve the backbone network of YOLOv8: Deformable convolution is applied to the backbone network of YOLOv8, called DeformBackbone. The network input size is set to 640×640. DeformBackbone has 5 convolutional layers Conv layers, 4 C2fs, and 1 SPPF. The convolutional layer types include conventional convolutional layers Conv and deformable convolutional layers Deform-Conv. The deformable convolutional layer Deform-Conv is set between every 2 C2Fs. The Bottleneck network in the C2F convolutional layer contains a deformable residual block, denoted as DeformBottleneck. The Bottleneck network consists of two 3×3 regular convolutional layers and one shortcut layer. When shortcut is True, DeformBottleneck replaces the two 3×3 regular convolutional layers with deformable convolutions, and batch normalization and activation functions are added after the deformable convolutional layers. S2.2, dual-modal feature fusion module; Phase 1: The RGB feature map F extracted by the two-stream backbone network RGB and the deep feature map F Depth Perform splicing operation to obtain the bimodal feature map F RGB-D , as shown in formula (1), F RGB-D =Concat(F RGB ;F Depth ) (1) Through the depth-wise separable convolution module DSConv 3*3 F RGB-D Perform convolution operations to integrate RGB information and depth information to construct correlation representation, and parallelize the patch-aware attention module PPA to capture feature information of different scales and levels through a multi-branch feature extraction fusion module and an adaptive attention mechanism module; By dividing the input features into p×p local patches, capturing local context information, and averaging the features within each patch, the parallelized patch-aware attention module screens important features through the multi-layer perceptron MLP and activation function Softmax, and uses the residual weighting method to fuse the local context, and finally generates the local feature F local and the global feature F glocal , the cascaded convolution consists of three 3×3 convolutional layers in series, which gradually fuse local and contextual information to obtain the serial convolution output F conv , the features of the above three branches are integrated to obtain a comprehensive feature representation As shown in formula (2): After multi-branch feature extraction, an adaptive attention mechanism is used to enhance adaptive channels and spatial dimensions. Sequentially, a one-dimensional channel attention map M C and a 2D spatial attention map M S Processing, the specific formula is as follows: in represents element-by-element multiplication, F C and F S Respectively represent the features after channel and space selection, δ, and They represent the rectified linear unit, batch normalization, and regularization function dropout, respectively, and F' is the final output of PPA; The second stage: construct the residual connection structure Res, filter redundant information, highlight key information, and construct the feature representation F' of the defect in three-dimensional space, as shown in formula (6). F" = F' + Res(F') (6).

4. The dual-modal target detection algorithm based on steel surface defects according to claim 3 is characterized in that: The multi-branch fusion in the multi-branch feature extraction fusion module includes patch perception and cascade convolution. The patch parameter p in patch perception is set to 2 or 4. When P is set to 2, it indicates a local branch, and when it is set to 4, it indicates a global branch. Different branches use patches of different sizes.

5. The dual-modal target detection algorithm based on steel surface defects according to claim 1, characterized in that: The step S3 specifically includes performing a power operation on the IoU, and the PloU loss function is expressed as: L PIoU =1-IoU p (p>1) (7) Among them, p is a hyperparameter used to balance the overlap between the predicted box and the true box; At different stages of training, the weights of PIoU and CloU are dynamically adjusted according to the IoU values ​​of the predicted box and the true box, combining the advantages of both. The combined loss function L CPIoU As shown in (8) L CPIoU =βL PIoU +(1-β)L CIoU (8) Where, L CIoU is the CIoU loss function, β is the dynamic weight, where β is adjusted according to the IoU value through the Sigmoid function; Among them, k is a hyperparameter that controls the speed of weight adjustment, and t is a hyperparameter that controls the threshold of weight adjustment.

6. The dual-modal target detection algorithm based on steel surface defects according to claim 1, characterized in that: The step S4 specifically includes, based on the collected data, customizing the data set to be divided into a variety of typical defect types, using the mean average precision mAP and the number of frames per second FPS as evaluation indicators, training the model, inputting the image data to be detected into the trained model described in step S2 for detection, and obtaining the detection results.

7. A system for dual-modal target detection algorithm based on steel surface defects according to any one of claims 1 to 6, characterized in that: It is a line structured light 3D measurement system, including a linear laser, a lens and a 3D camera; The linear laser is used to make the light knife incident vertically onto the steel surface; The 3D camera is used to obtain a laser cross-sectional image of the steel component; The lens is used to focus the light knife emitted by the linear laser and project it onto the surface of the rail, and to image the modulated light strip onto the sensor of the 3D camera.

Images