PCAR-YOLO-based steel surface defect lightweight detection method and device

Abstract

The invention belongs to the technical field of defect detection, and particularly relates to a PCAR-YOLO-based steel surface defect lightweight detection method and device. Comprising the following steps: S1, carrying out standardized preprocessing on an acquired steel surface image; s2, inputting the image data into a feature extraction network, wherein the feature extraction network comprises a multi-dimensional dynamic convolution module; s3, performing multi-scale feature extraction on the image data through a PentDConv module; s4, inputting the enhanced defect feature representation into a content awareness feature recombination network; and S5, performing content-aware up-sampling and high-fidelity detail reconstruction on each scale feature map by using the ICARAFE module, and fusing the up-sampled multi-scale features through a feature fusion network with a cross-scale connection structure. And S6, performing defect identification and positioning through the detection head network. According to the method, a multi-dimensional dynamic feature enhancement convolution module PentDConv is provided, and the feature expression capability is remarkably improved under the condition that the calculation amount is hardly increased.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, and in particular relates to a lightweight detection method and device for steel surface defects based on PCAR-YOLO. Background Technology

[0002] The Industrial Internet of Things (IIoT) provides real-time, high-quality steel surface image data for smart manufacturing by deploying visual sensing devices. Intelligent inspection systems then feed the defect identification results back to the production control system, enabling online monitoring of product quality and process optimization. The inspection system intelligently analyzes the acquired steel surface images to identify and locate various surface defects. However, steel surface defects in complex industrial environments exhibit multi-scale, low-contrast, and diverse morphological characteristics, posing a significant challenge to high-precision defect detection.

[0003] The technological evolution of steel surface defect detection can be divided into three typical stages: the traditional digital image processing stage, the machine learning-driven stage, and the deep learning stage. The traditional digital image processing stage is mainly based on morphological operations and texture analysis. Morphological operations, based on set theory and topology, perform operations such as dilation, erosion, opening, and closing on images using structural elements such as circular and rectangular kernels. Texture analysis feature extraction methods include gray-level co-occurrence matrix, local binary pattern, and Gabor filtering. However, the traditional digital image processing stage is limited by its reliance on manually designed feature extraction rules, which are difficult to adapt to complex working conditions, especially when the defect size is similar to the texture period, leading to a sharp increase in the false positive rate. The machine learning-driven stage decouples feature engineering from classification algorithms, employing classification algorithms such as support vector machines and random forests to automatically learn defect features. It establishes a more robust defect representation system through methods such as histograms of oriented gradients. Simultaneously, the introduction of semi-supervised and unsupervised learning methods alleviates the problem of scarce labeled samples in industrial scenarios. However, the methods in the machine learning-driven stage depend on the quality of feature engineering and have limited generalization ability, especially exhibiting unstable performance in small sample scenarios. Deep learning stage: An end-to-end neural network architecture is adopted to realize the direct mapping from the original image to defect recognition and to automatically learn multi-level and abstract defect feature representation. However, this strategy has problems such as high computational complexity and weak real-time performance of deep neural networks, high false negative rate of existing algorithms for small defects, and difficulty in effectively modeling multi-scale and multi-morphological defect features.

[0004] To address the shortcomings of existing deep learning methods in industrial defect detection, particularly their low accuracy in detecting small targets, weak detail preservation, and low computational efficiency, this invention proposes a lightweight defect detection method that integrates five-dimensional dynamic convolution and content-aware upsampling. This method aims to solve technical challenges in industrial environments, such as the difficulty in balancing accuracy and efficiency in detection models and insufficient sensitivity to small target defects. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a lightweight detection method and device for steel surface defects based on PCAR-YOLO. Applied to online monitoring of industrial steel surface quality, it significantly improves the detection rate and location accuracy of minute defects.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a lightweight detection method for steel surface defects based on PCAR-YOLO, comprising the following steps: S1. Standardize and preprocess the collected steel surface images to form image data suitable for deep network analysis.

[0007] S2. Input the image data into the feature extraction network, which includes a multidimensional dynamic convolution module, abbreviated as PentDConv module. The PentDConv module is used to adaptively adjust the convolution kernel parameters based on the input image content.

[0008] S3. Perform multi-scale feature extraction on the image data through the PentDConv module to obtain enhanced defect feature representation.

[0009] S4. The enhanced defect feature representation is input into a content-aware feature reconstruction network, which includes an improved content-aware feature reconstruction module, abbreviated as ICARAFE module. The ICARAFE module is used to dynamically reconstruct the spatial resolution and details of the feature map according to the semantic importance of the features.

[0010] S5. Input the multi-scale defect feature map extracted by the PentDConv module into the ICARAFE module. Use the ICARAFE module to perform content-aware upsampling and high-fidelity detail reconstruction on the feature maps of each scale. Then, fuse the upsampled multi-scale features through a feature fusion network with a cross-scale connection structure to obtain the fused multi-scale defect feature map.

[0011] S6. Based on the fused multi-scale defect feature map, the defect is identified and located through the detection head network, and the category and location information of the steel surface defect is output.

[0012] Further, S1 includes: S11. Divide the original image into fixed sizes and extract local image blocks using a sliding window method. The window size is set to N×N pixels, and p% of the overlapping area is retained between adjacent windows, where p is a real number greater than 0.

[0013] S12. Perform brightness normalization, contrast stretching, and Gaussian filtering on the divided image blocks in sequence.

[0014] S13. Apply the Adaptive Histogram Equalization (CLAHE) algorithm to enhance local image contrast.

[0015] S14. Scale the enhanced image to the standard input size.

[0016] S15. Perform data augmentation operations on the image, including random flipping, rotation, scaling, brightness perturbation, and Gaussian noise injection.

[0017] S16. Divide the processed images into training set, validation set and test set according to the proportion.

[0018] Furthermore, the PentDConv module constructed in S2 generates dynamic convolutional kernels through the following steps: S21. Extract global information from the input features to obtain a global context representation of the channel dimension.

[0019] S22. The global context representation is reduced in dimensionality and nonlinearly mapped through a feature compression layer.

[0020] S23, Based on the preset number of groups G Divide the input feature channels into G Each subchannel group contains [number] independent subchannel groups, and each subchannel group contains [number] independent subchannel groups. C IN / G A series of continuous channels enable the separation and interaction of local and global features.

[0021] S24. Attention weights are generated for the filter, spatial, channel, and convolutional kernel dimensions through four independent attention branches. The channel-dimensional attention weights are specifically designed for... G Each sub-channel group is generated separately to differentiate and enhance sub-channel groups dominated by defect features and suppress sub-channel groups dominated by background texture. C IN This represents the number of channels in the input feature map.

[0022] S25. The attention weights are weighted and fused with predefined candidate convolutional kernels to generate dynamic convolutional kernels that adapt to the input content.

[0023] Further, S3 includes: S31. Scale the preprocessed image and fill it to the preset size to form an input tensor.

[0024] S32. Perform preliminary feature extraction and downsampling through basic convolutional layers.

[0025] S33. Input the feature map into the PentDConv module and use the dynamic convolution kernel generated by it to perform convolution operation to extract dynamic features.

[0026] S34. The extracted features are sequentially passed through the C3 module, which consists of multiple Bottleneck residual structures, the convolutional downsampling layer, and the SPPF module to gradually extract deeper features.

[0027] Furthermore, the ICARAFE module constructed in S4 achieves content-aware upsampling through the following steps: S41. The input features are reduced in dimensionality and encoded by the kernel prediction module to generate kernel prediction features.

[0028] S42. The kernel prediction features are rearranged to an upsampled resolution using the PixelShuffle operation.

[0029] S43. Normalize the rearranged features using the Softmax function in the channel dimension to obtain a spatially adaptive upsampling kernel tensor.

[0030] S44. Through the content-aware reconstruction module, the input features are filled, local feature blocks are extracted, and matrix multiplication is performed to reconstruct the matrix using the upsampled kernel tensor, thereby achieving high-fidelity feature reconstruction.

[0031] Further, S5 includes: S51. Input the multi-scale feature map extracted by the PentDConv module into the ICARAFE module.

[0032] S52. Using the upsampled feature map output by the ICARAFE module, semantic and detailed information at different levels are fused through a feature fusion network with a cross-scale connection structure.

[0033] Further, S6 includes: S61. The fused multi-scale feature map is input into three detection heads of different scales. The three detection heads process feature maps of different resolutions: 80×80×256, 40×40×512, and 20×20×1024, respectively, to optimize the detection of defects with bounding box areas smaller than 32×32 pixels, between 32×32 and 96×96 pixels, and larger than 96×96 pixels.

[0034] S62. Preset a set of reference boxes of different sizes on the feature map at each scale, and normalize the predicted target location parameters.

[0035] S63. Predict the probability of defect categories through classification branches and predict the bounding box coordinates through regression branches.

[0036] S64. Combine the classification probability and the target existence probability to obtain the final detection confidence.

[0037] S65. Use a composite loss function that includes classification loss, bounding box regression loss, and object-related loss for model training.

[0038] S66. Perform threshold filtering and non-maximum suppression (NMS) processing on the prediction results to remove redundant detection boxes and output the defect category label, confidence level and location coordinates.

[0039] The PCAR-YOLO-based method for lightweight detection of surface defects in steel, wherein S1, the standardization preprocessing of the acquired steel surface image, further includes: Lens distortion correction and brightness non-uniformity correction are performed on the acquired steel surface images to eliminate the influence of geometric distortion and uneven illumination introduced by the imaging system on the defect detection results.

[0040] The Region of Interest (ROI) is determined based on the geometric contour of the steel and the production process parameters. The ROI is then cropped in the corrected image, retaining only the effective region containing the steel surface as input for subsequent feature extraction.

[0041] Furthermore, the image acquisition and preprocessing module is used to perform the method step S1 of any one of claims 1 to 8, to perform standardized preprocessing on the acquired steel surface image to form image data suitable for deep network analysis.

[0042] The feature extraction module includes a PentDConv module for executing method steps S2 and S3 as described in any one of claims 1 to 8, receiving the image data and performing multi-scale feature extraction to obtain an enhanced defect feature representation.

[0043] The feature recombination and fusion module, including the ICARAFE module and the feature fusion network, is used to execute the method steps S4 and S5 of any one of claims 1 to 8, performing content-aware upsampling, high-fidelity detail reconstruction and cross-scale feature fusion on the enhanced defect feature representation to obtain a fused multi-scale defect feature map.

[0044] The defect detection and output module is used to execute the method step S6 of any one of claims 1 to 8, to identify and locate defects based on the fused multi-scale defect feature map, and to output the category and location information of the defects on the steel surface.

[0045] Furthermore, the detection device is deployed on an industrial computer or embedded AI computing platform and communicates with industrial cameras, light source controllers and production line PLCs to achieve real-time online defect detection and sorting control of moving steel surfaces on the steel production line.

[0046] Compared with the prior art, the present invention has the following advantages.

[0047] This invention proposes a multidimensional dynamic feature enhancement convolution module, PentDConv, which constructs a dynamic weight generation system integrating spatial, channel, convolutional kernel, grouping, and filter dimensions. It generates low-dimensional feature representations by compressing input features through convolution, and then synchronously calculates the dynamic weights of each dimension. These weights are fused with the initial convolutional kernel to finally generate a dynamic convolutional kernel that adapts to the input features, which significantly improves feature representation capabilities with almost no increase in computation.

[0048] This invention designs a semantically aware upsampling strategy and proposes an ICARAFE upsampling method. ICARAFE adaptively adjusts the upsampling weights based on local feature content, transforming the upsampling process from a fixed interpolation operation to a content-aware dynamic convolution operation. It utilizes multi-scale dynamic features extracted by PentDConv as contextual priors and uses a kernel prediction network to generate spatially adaptive upsampling kernels, achieving high-fidelity reconstruction of defect edge details.

[0049] This invention designs a novel defect detection model, PCAR-YOLO, which improves the model's feature extraction capabilities for complex textures and minute defects by introducing a designed five-dimensional dynamic feature enhancement convolution module. Furthermore, it uses the ICARAFE upsampling method to enhance the spatial detail restoration capability of small target defects. While maintaining high accuracy, it also takes into account lightweight design and real-time performance, significantly improving the balance between sensitivity to small targets and computational efficiency in the detection of surface defects in industrial steel. Attached Figure Description

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0051] Figure 1 This is a system structure overview diagram of the present invention; Figure 2 This is a schematic diagram of the system structure of the present invention; Figure 3 This is a schematic diagram of the method framework of the present invention; Figure 4 This is a schematic diagram of the content-aware upsampling of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0053] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” as used in the embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0054] Depending on the context, words such as “if” or “suppose” used here can be interpreted as “when”, “in response to determination”, or “in response to detection”.

[0055] For ease of understanding, the embodiments of this disclosure will be described in detail first.

[0056] The PCAR-YOLO-based method for lightweight detection of steel surface defects in this embodiment includes: S1: Image Acquisition and Preprocessing: Images of the steel surface are acquired using a camera system, and then subjected to noise reduction, normalization, and cropping. Image preprocessing ensures image quality and standardization, making it suitable for subsequent feature extraction.

[0057] S2: Feature Extraction: The acquired images are input into the five-dimensional dynamic convolution module (PentDConv) for feature extraction. PentDConv utilizes a five-dimensional attention mechanism to dynamically aggregate multi-scale convolution kernel features, enhancing the expression of local defect features and improving the detection capability of minute defects; S3: Multi-scale feature fusion: In the feature fusion layer, a high-fidelity reconstruction of multi-scale features is performed by improving the content-aware feature reconstruction (ICARAFE) operator. ICARAFE dynamically adjusts the upsampling weights based on local features, effectively preserving the edge details of defects on the steel surface and optimizing the spatial detail recovery of small target defects.

[0058] S4: Defect Detection and Classification: The model outputs the final defect category and location through multiple scale detection heads. It employs a Feature Pyramid Network (FPN) and a top-down feature fusion strategy to enhance the model's adaptability to defects of both large scale and small target size, enabling accurate defect classification and localization.

[0059] Preferably, S2: Feature extraction: The acquired images are input into the five-dimensional dynamic convolution module (PentDConv) for feature extraction. PentDConv uses a five-dimensional attention mechanism to dynamically aggregate multi-scale convolution kernel features, enhance the expression of local defect features, and improve the detection capability of small defects.

[0060] S3: Multi-scale feature fusion: In the feature fusion layer, a high-fidelity reconstruction of multi-scale features is performed by improving the Content-Aware Feature Reconstruction (ICARAFE) operator. ICARAFE dynamically adjusts the upsampling weights based on local features, effectively preserving the edge details of defects on the steel surface and optimizing the spatial detail recovery of small target defects; S4: Defect Detection and Classification: The model outputs the final defect category and location through multiple scale detection heads. It employs a Feature Pyramid Network (FPN) and a top-down feature fusion strategy to enhance the model's adaptability to defects of both large scale and small target size, enabling accurate defect classification and localization.

[0061] Preferably, the specific steps of S1 are as follows: S101. Divide the acquired original steel surface image into fixed sizes, and extract local image blocks along the spatial dimension using a sliding window method. Set the window size to N×N, and retain p% of the overlapping area between adjacent windows to ensure the continuity of edge features and detection stability.

[0062] S102. Perform image enhancement processing on the divided image blocks, specifically including brightness normalization, contrast stretching, and Gaussian filtering for noise reduction. The normalization calculation formula is as follows:

[0063] Where I(x, y) is the pixel grayscale value of the original image, I min and I max These represent the minimum and maximum gray levels in the image, respectively. This step eliminates the influence of illumination changes on the detection results and improves the model's robustness to different batches of images.

[0064] S103. Apply the Adaptive Histogram Equalization (CLAHE) algorithm to the normalized image to enhance the contrast of minor surface defects. The calculation formula is as follows:

[0065] in, CDF Let be the cumulative distribution function. L This step involves setting grayscale levels. It ensures that the dynamic range of brightness in local image regions is enhanced, thereby highlighting defect details.

[0066] S104. Perform size unification processing on the enhanced images, scaling all images to the standard input size of W×H, and using a bilinear interpolation algorithm. The calculation formula is as follows:

[0067] in, The coordinates of the original image and the target image The nearest neighbor pixel.

[0068] S105. Perform data augmentation operations on the preprocessed image, including random flipping, rotation, scaling, brightness perturbation, and Gaussian noise injection. The rotation transformation formula is as follows:

[0069] in, The rotation angle is within a certain range. This step is used to improve the model's generalization ability under different shooting angles and lighting conditions.

[0070] S106. Store all processed images in a uniform format and divide them into training, validation, and test sets in a ratio of 8:1:1 to provide standardized input data for feature extraction and training of the PCAR-YOLO model.

[0071] Preferably, the specific steps of S2 are as follows: the feature extraction algorithm is based on the PyTorch deep learning framework, and the PCAR-YOLO model is trained end-to-end on the NEU-DET and GC10-DET steel surface defect datasets.

[0072] S201. Scale and fill the steel surface image to 640×640 pixels according to the standard network input size, and then normalize and correct the channel order to form a tensor. As input to the backbone network, it ensures that multi-scale features are aligned on a unified grid and mitigates the impact of illumination differences.

[0073] S202. The image first undergoes preliminary feature extraction through a basic convolutional layer (Conv). This layer uses a 6×6 convolutional kernel, with 64 output channels, a stride of 2, and padding of 2 for downsampling. The final output size is 320×320×64. S203. The feature map is generated through the PentDConv module, which is an improved five-dimensional dynamic convolution module. The dynamic convolution kernel is generated through the following steps: S2031. Perform global average pooling on the input features to obtain the global context representation of the channel dimension; the output size is 1×1×128.

[0074] S2032. The global context representation is reduced in dimensionality and nonlinearly mapped using a 1×1 convolutional layer, wherein the hidden layer dimension... d h Calculated using the following formula:

[0075] Where `max` is the maximum value function, `Cin` is the number of channels in the input feature map, and `r` is the compression ratio, which defaults to 1 / 16 and has a lower limit of 16. When based on... When the calculated dimension is less than 16, The value is forced to 16 to ensure that the hidden layer has the most basic representation capabilities.

[0076] S2033. Through four independent fully connected layers, attention weights are generated in the filter dimension, spatial dimension, channel dimension and convolutional kernel dimension respectively; thus realizing multi-dimensional dynamic control of the convolutional kernel.

[0077] S2034. The attention weights are weighted and fused with K predefined candidate convolutional kernels to generate a dynamic convolutional kernel W that adapts to the input content. d The mathematical expression is:

[0078] in As candidate convolution kernels, The attention weights are the convolution kernel dimension. This represents the integrated attention weight after fusion.

[0079] S204. The feature map is passed through the C3 module, which consists of the Bottleneck residual structure. It enhances feature reuse through cross-layer connections, and the output size remains unchanged at 160×160×128. S205. The feature map passes through the third Conv layer, which uses a 3×3 convolutional kernel with 256 output channels and a stride of 2 for downsampling. The final output size is 80×80×256. S206. The feature map is processed through the C3 module, which contains 6 Bottleneck residual structures to further extract deep features, and the output size is maintained at 80×80×256.

[0080] S207. The feature map passes through the fourth Conv layer, which uses a 3×3 convolution kernel with 512 output channels and a stride of 2 for downsampling. The final output size is 40×40×512.

[0081] S208. The feature map is processed through the C3 module, which contains 9 Bottleneck residual structures to extract more complex feature representations, while maintaining the output size of 40×40×512.

[0082] S209. The feature map passes through the fifth layer Conv, which uses a 3×3 convolutional kernel with 1024 output channels and a stride of 2 for downsampling. The final output size is 20×20×1024.

[0083] S210, the feature map is passed through the C3 module, which contains 3 Bottleneck residual structures, and the output size is kept at 20×20×1024.

[0084] S201. The feature map is processed by the SPPF module, which adopts a serial max pooling strategy. It uses a 5×5 pooling kernel to perform three serial pooling operations to capture multi-scale receptive field features. The final output size is 20×20×1024.

[0085] Preferably, the specific steps of multi-scale feature fusion in S3 are as follows: S301. The multi-scale dynamic features extracted by the PentDConv module are used as contextual prior information and input into the ICARAFE upsampling module. The shape of the input feature tensor is... Where N is the batch size, C is the number of channels, and H and W are the space dimensions.

[0086] The S302 and ICARAFE modules first generate a spatially adaptive upsampling kernel through the kernel prediction module. Given an input feature tensor X, this module first reduces the number of channels through dimensionality-reducing convolutions, generating an intermediate feature tensor after 1×1 convolution dimensionality reduction. .

[0087] S303, Generating kernel prediction features through encoder convolution. X That is, the constructed ICARAFE module, whose kernel prediction module generates kernel prediction features K through encoder convolution. pre :

[0088] in This is the reduced-dimensionality input feature tensor, derived from the output of the previous 1×1 convolution. N is the batch size, representing the number of feature maps processed in a single operation. The encoder convolution kernel size, The upsampling factor, is the size of the upsampling kernel. H and W are the spatial height and width of the input feature map, respectively.

[0089] S304. Use the PixelShuffle operation to rearrange the kernel prediction features to an upsampled resolution.

[0090] S305. Normalization is achieved through Softmax in the channel dimension to obtain a kernel tensor that satisfies the weighted constraints.

[0091] S306. The content-aware reconstruction module reconstructs the input features using the predicted checksums. First, the input features are filled in.

[0092] S307. Extract local feature blocks using a sliding window operation to obtain a shape of The feature tensor.

[0093] S308, the kernel tensor, after rearrangement and expansion operations, has the following final shape: .

[0094] S309. Content-aware reorganization is achieved through matrix multiplication.

[0095] S310. Restore spatial resolution through pixel rearrangement and output the upsampled feature map. 。

[0096] S311, ICARAFE, and the PentDConv module work together to fully utilize the multi-scale dynamic features extracted by PentDConv as contextual prior information. The mathematical expression of the output features of PentDConv is as follows:

[0097] Each of them These represent characteristic responses at different scales.

[0098] S312. Multi-scale feature fusion is performed through the Feature Pyramid Network (FPN) structure. A top-down feature propagation and lateral connection strategy is adopted to effectively fuse high-level semantic features with low-level spatial detail features, forming a comprehensive feature representation that has both strong semantic expression and high spatial resolution.

[0099] Preferably, the specific steps for defect detection and classification in S4 are as follows: S401. Input the three scale feature maps output from the multi-scale feature fusion stage into the corresponding detection heads. The three detection heads process feature maps of different resolutions: 80×80×256, 40×40×512, and 20×20×1024, respectively, to optimize the detection of defects with bounding box areas less than 32×32 pixels, between 32×32 and 96×96 pixels, and greater than 96×96 pixels.

[0100] S402. Predefine several anchor boxes on each scale feature map, each anchor box being represented by its center point coordinates and width and height parameters:

[0101] in, It represents the relative coordinates of the anchor frame center point within its corresponding feature map grid cell, with values ​​ranging from [0,1]. These are the width and height of the anchor frame, whose values ​​are pre-set absolute values ​​based on the current inspection scale. Multiple anchor frames with different aspect ratios are assigned to each location to cover defect areas of varying shapes and proportions. The predicted offset is normalized using the Sigmoid function, enabling dynamic adjustment of the anchor frame center and dimensions to align it with the actual defect location.

[0102] S403. Predict using a classification and regression branch detection head. The classification branch is used to determine which type of defect the target belongs to. The output class probability is obtained through multiple 1×1 convolutional layers and sigmoid activation.

[0103] in, This is a category probability vector, where each element represents the predicted probability of the corresponding defect category. It is the Sigmoid activation function, used to normalize the output to probability values. Y is the feature map input to the detector head, and Wc and b are... c These are the weights and bias parameters of the classification branch convolutional layer.

[0104] The regression branch is used to predict the center offset and aspect ratio of the target bounding box, and outputs a four-dimensional coordinate regression vector:

[0105] in, It is the offset of the center point of the predicted bounding box relative to the center point of the anchor box. This is the scaling ratio of the predicted bounding box's width and height relative to the anchor box's width and height. The predicted bounding box position is then restored to its actual pixel coordinates using a decoding function, achieving precise positioning.

[0106] S404. The confidence scores output by each detection branch are fused to form the final detection confidence score S, calculated using the following formula:

[0107] Among them, P c It is the maximum class probability output by the classification branch, i.e., max(P) c ), P obj This indicates the probability that the target exists.

[0108] The loss function consists of three parts:

[0109] in, Represents classification loss. This represents the bounding box regression loss. Indicates object-oriented loss. This represents the balancing weighting coefficient, used to adjust the contribution ratio of the three losses to the total loss.

[0110] S405. The detection results are filtered by threshold and redundant boxes are removed using the NMS algorithm. For the same defect area, only the detection result with the highest confidence is retained. The final output includes the defect category, confidence level, and location coordinates, forming:

[0111] Where C i For defect category, S i To test the confidence level. Let be the absolute coordinates of the center point of the i-th detection box on the original input image. Let represent the absolute width and height of the i-th detection box on the original input image, and N represent the final number of detection boxes retained after NMS.

[0112] S406. Model Training and Optimization. During model training, the SGD optimizer is used with an initial learning rate of 0.01. A cosine annealing scheduling strategy is employed to gradually decrease the learning rate. To enhance the model's robustness, Mosaic data augmentation is introduced.

[0113] like Figure 1 As shown in the figure, the system structure overview diagram of the lightweight detection method and device for steel surface defects based on PCAR-YOLO is as follows. Figure 1 As shown, the system comprises an image acquisition unit, a data processing unit, and a system deployment and communication unit. The image acquisition unit acquires images of the steel surface on the production line and synchronizes exposure and triggering, providing stable input for subsequent detection algorithms. The data processing unit is responsible for feature extraction, feature fusion and reconstruction, and outputting defect categories and locations, providing core algorithmic support for high-precision and real-time identification of small-target defects on the steel surface. The system deployment and communication unit is responsible for model inference, version and log management at the edge, and interfacing with the upper-level system via Modbus to achieve result reporting and alarm linkage, providing reliable integration assurance for stable production line operation and quality traceability.

[0114] Figure 2The diagram shows the system structure of this invention. A lightweight steel surface defect detection device based on PCAR-YOLO includes an image acquisition section, a data processing section, and a system deployment and communication section. The image acquisition section mainly includes an image acquisition module, an illumination module, a synchronization control module, and a mechanical stabilization module. The image acquisition module is responsible for acquiring high-resolution images of the steel surface under production line cycle time. The illumination module is responsible for providing a uniform and adjustable incident angle light source, suppressing reflections and shadows, and enhancing defect contrast. The synchronization control module is responsible for receiving cycle signals from photoelectric sensors, hardware-triggered camera exposure, eliminating motion blur, and ensuring strict synchronization with the conveyor speed. The mechanical stabilization module is responsible for the rigid fixation and vibration reduction design of the camera, lens, light source, and support to avoid imaging drift caused by micro-vibrations. The illumination module, mechanical stabilization module, and synchronization control module assist the image acquisition module in image capture. The data processing section mainly includes a data preprocessing module, a feature extraction module, a feature reconstruction module, a multi-scale fusion module, a detection module, and a post-processing module. The data preprocessing module is responsible for distortion and brightness correction, normalization, ROI cropping, and denoising to form the standard input for the algorithm. The feature extraction module replaces part of the standard convolution with five-dimensional dynamic convolution, and dynamically generates and selects weights by combining spatial, channel, convolution kernel, grouping, and filter dimensions to enhance the discrimination features of complex textures and minor defects while controlling the computational load. The feature reconstruction module uses local content prediction adaptive upsampling kernels to achieve high-fidelity reconstruction of details and edges, reducing the loss of details in small targets and low-contrast defects. The multi-scale fusion module fuses semantic and spatial information at different scales through fast spatial pyramid pooling and top-down and lateral connections to enhance multi-scale target representation. The detection module outputs the category, bounding box, and confidence score at three feature map scales: 80×80 pixels, 40×40 pixels, and 20×20 pixels. The post-processing module performs threshold filtering and NMS to generate the final defect location information and category results. The system deployment and communication part includes an edge inference module, a model management module, a data transmission module, and a monitoring and alarm module. The edge inference module is responsible for performing model inference on industrial edge devices to achieve real-time defect detection and classification. The model management module is responsible for model loading, updating, version management, and lightweight deployment. The data transmission module is responsible for uploading the detection results to the host computer in real time via protocols such as Modbus. The monitoring and alarm module is responsible for monitoring the system status in real time, providing timely feedback on operational anomalies and detection performance, and triggering alarm mechanisms.

[0115] Figure 3 The diagram shown is a schematic representation of the method framework of this invention. Figure 3(a) represents image preprocessing, ensuring image quality and standardization for subsequent feature extraction. The first step involves dividing the acquired raw steel surface image into fixed-size blocks and extracting local image patches along the spatial dimension using a sliding window approach. The window size is set to N×N, with p% overlap between adjacent windows to ensure the continuity of edge features and detection stability. Then, image enhancement processing is performed on the divided image patches, specifically including brightness normalization, contrast stretching, and Gaussian filtering for noise reduction. The normalization calculation formula is as follows:

[0116] Where I(x, y) is the pixel grayscale value of the original image, I min and I max These represent the minimum and maximum gray levels in the image, respectively. This step eliminates the influence of illumination changes on the detection results and improves the model's robustness to different batches of images.

[0117] Then, an adaptive histogram equalization (CLAHE) algorithm is applied to the normalized image to enhance the contrast of subtle surface defects. The calculation formula is as follows:

[0118] Where CDF is the cumulative distribution function and L is the number of gray levels. This step ensures that the dynamic range of brightness in local areas of the image is enhanced, thereby highlighting defect details.

[0119] Then, the enhanced images are scaled down to a uniform size. The standard input size is calculated using a bilinear interpolation algorithm, and the formula is as follows:

[0120] Among them, (x) i y j (x, y) represents the pixel in the original image that is closest to the target coordinates (x, y).

[0121] The preprocessed image is then subjected to data augmentation operations, including random flipping, rotation, scaling, brightness perturbation, and Gaussian noise injection. The rotation transformation formula is as follows:

[0122] in, The rotation angle is within a certain range. This step is used to improve the model's generalization ability under different shooting angles and lighting conditions.

[0123] Finally, all processed images are stored in a uniform format and divided into training, validation, and test sets in a ratio of 8:1:1, providing standardized input data for feature extraction and training of the PCAR-YOLO model.

[0124] Figure 3 (b) indicates that the acquired image is input into the five-dimensional dynamic convolution module (PentDConv) for feature extraction. Through the five-dimensional dynamic convolution mechanism and SPPF multi-scale pooling operation of the PentDConv module, the image data is extracted into feature representations at three scales: 20×20×1024, 40×40×512, and 80×80×256. A multi-scale feature pyramid is established to enhance the perception capability of defects of different sizes.

[0125] Figure 3 (c) indicates that in the feature fusion layer, the ICARAFE operator is used to perform high-fidelity reconstruction of multi-scale features. ICARAFE dynamically adjusts the upsampling weights based on local features, effectively preserving edge details of steel surface defects and optimizing the spatial detail recovery of small target defects. Specifically, the multi-scale dynamic features extracted by the PentDConv module are used as contextual prior information and input into the ICARAFE upsampling module. The shape of the input feature tensor is... Where N is the batch size, C is the number of channels, and H and W are the spatial dimensions. The ICARAFE module then first generates a spatially adaptive upsampling kernel through a kernel prediction module. Given an input feature tensor X, this module first reduces the number of channels through dimensionality-reducing convolutions, generating an intermediate feature tensor X after 1×1 convolution dimensionality reduction. d Then, kernel prediction features are generated through encoder convolution. X ,

[0126] Where Xd is the dimensionality-reduced input feature tensor, derived from the output of the previous 1×1 convolution, N is the batch size, representing the number of feature maps processed at once, k=5 is the encoder convolution kernel size, and s=2 is the upsampling factor. up Let H be the size of the upsampling kernel. H and W are the spatial height and width of the input feature map, respectively. The kernel-predicted features are rearranged to the upsampling resolution using the PixelShuffle operation. Then, normalization is achieved through Softmax along the channel dimension to obtain a kernel tensor that satisfies the weighted constraints. The content-aware reconstruction module then uses the predicted kernel to reconstruct the input features. First, the input features are padded. Local feature blocks are extracted using a sliding window operation to obtain a shape... The feature tensor. Then the kernel tensor undergoes rearrangement and expansion operations, and the final shape is... Content-aware reassembly is achieved through matrix multiplication. Spatial resolution is restored through pixel rearrangement, and the upsampled feature map is output. ICARAFE works in collaboration with the PentDConv module, fully utilizing the multi-scale dynamic features extracted by PentDConv as contextual prior information. The mathematical expression of the output features of PentDConv is as follows:

[0127] Each f i The feature responses represent different scales. Finally, multi-scale feature fusion is performed through a Feature Pyramid Network (FPN) structure. A top-down feature propagation and lateral connection strategy is adopted to effectively fuse high-level semantic features with low-level spatial detail features, forming a comprehensive feature representation that has both strong semantic expression and maintains high spatial resolution.

[0128] Figure 3 (d) represents defect detection and classification, where the three scale feature maps output from the multi-scale feature fusion stage are input into the corresponding detection heads. The three detection heads process feature maps of different resolutions: 80×80×256, 40×40×512, and 20×20×1024, respectively optimizing the detection of defects with bounding box areas less than 32×32 pixels, between 32×32 and 96×96 pixels, and greater than 96×96 pixels. Then, several anchor boxes are predefined on each scale feature map, each represented by its center point coordinates and width and height parameters.

[0129] Multiple anchor frames with different aspect ratios are assigned at each location to cover defect areas of varying shapes and proportions. The predicted offsets are normalized using a sigmoid function, enabling dynamic adjustment of the anchor frame center and size to align them with the actual defect locations. Then, a classification and regression branch detection head predicts the defect type. The classification branch determines the defect category by using multiple 1×1 convolutional layers and a sigmoid activation function to output the category probability.

[0130] Among them, P c This is a category probability vector, where each element represents the predicted probability of the corresponding defect category. It is the Sigmoid activation function, used to normalize the output to probability values. Y is the feature map input to the detector head, and Wc and b are... c These are the weights and bias parameters of the classification branch convolutional layer.

[0131] The regression branch is used to predict the center offset and aspect ratio of the target bounding box, and outputs a four-dimensional coordinate regression vector:

[0132] Among them, t x ,t y It is the offset of the center point of the prediction box relative to the center point of the anchor box, t w ,t h This is the scaling ratio of the predicted bounding box's width and height relative to the anchor box's width and height. The predicted bounding box position is restored to its actual pixel coordinates through a decoding function, achieving precise localization. The confidence scores output by each detection branch are fused to form the final detection confidence score S, calculated using the following formula:

[0133] Among them, P c It is the maximum class probability output by the classification branch, i.e., max(P) c ), P obj This indicates the probability that the target exists.

[0134] The loss function consists of three parts:

[0135] in, Represents classification loss. This represents the bounding box regression loss. Indicates object-oriented loss. This represents the balancing weighting coefficient, used to adjust the contribution ratio of the three losses to the total loss.

[0136] Finally, the detection results are thresholded and redundant boxes are removed using the NMS algorithm. For the same defect area, only the detection result with the highest confidence is retained. The final output includes the defect category, confidence level, and location coordinates, forming:

[0137] Where C i For defect category, S i To test the confidence level. x i ,y i Let w be the absolute coordinates of the center point of the i-th detection box on the original input image. i ,h i Let represent the absolute width and height of the i-th detection box on the original input image, and N be the final number of detection boxes retained after NMS. During model training, the SGD optimizer is used with an initial learning rate of 0.01. A cosine annealing scheduling strategy is employed to gradually decay the learning rate. To enhance the model's robustness, Mosaic data augmentation is introduced.

[0138] Figure 4 The diagram shown illustrates the multi-scale feature fusion process of this invention. The specific process is as follows: The first part is the data flow path of the kernel prediction module, which is responsible for generating a spatially adaptive upsampling kernel that adapts to the input content. First, the input shape is... The feature tensor is defined as follows: N is the batch size, C is the number of channels, and H and W are the spatial dimensions. After passing through a channel compressor, the number of channels is reduced from C to [value missing] using dimensionality reduction convolution. , The channel compression coefficient is used to reduce the computational complexity of subsequent operations; then, the content encoder processes the data to generate kernel prediction features. Its shape is S=2 is the upsampling factor. The first part defines the upsampling kernel size. Then, the kernel prediction features are rearranged according to the upsampling resolution using a PixelShuffle operation to align the spatial dimension with the input features. Finally, Softmax normalization is performed on the channel dimension to obtain a kernel tensor that satisfies weighted constraints, completing the generation of the adaptive upsampling kernel. The second part is the data flow path of the content-aware reconstruction module, which uses the kernel tensor generated in the first part to perform high-fidelity reconstruction of the input features. First, edge padding is performed on the original input features to avoid loss of edge information during sliding window operations. Then, local feature blocks are extracted through a sliding window to obtain a shape... The local feature tensor; simultaneously, the kernel tensor output from the first part is rearranged and expanded, adjusted to... The shape is determined to match the local feature tensor in dimension; then, content-aware recombination of the kernel tensor and local feature blocks is achieved through matrix multiplication, so that the feature recombination process fits the local details of the defect edge on the steel surface; finally, spatial resolution is restored through pixel rearrangement operation, and the upsampled features are output, whose spatial size is s times that of the input features, while retaining the edge and detail information of the defect.

[0139] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "preferred embodiment," "detailed description," or "preferred embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0140] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Therefore, these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the scope defined by the claims of the present invention.

Claims

1. A PCAR-YOLO-based lightweight detection method for steel surface defects, characterized in that, The method comprises the following steps: S1, standardizing and preprocessing the collected steel surface image to form image data suitable for deep network analysis; S2, inputting the image data into a feature extraction network, wherein the feature extraction network comprises a multi-dimensional dynamic convolution module, referred to as a PentDConv module, and the PentDConv module is used to adaptively adjust the convolution kernel parameters based on the input image content; S3, performing multi-scale feature extraction on the image data through the PentDConv module to obtain enhanced defect feature representation; S4, inputting the enhanced defect feature representation into a content-aware feature reorganization network, wherein the content-aware feature reorganization network comprises an improved content-aware feature reorganization module, referred to as an ICARAFE module, and the ICARAFE module is used to dynamically reconstruct the spatial resolution and details of the feature map according to the semantic importance of the features; S5, inputting the multi-scale defect feature map extracted by the PentDConv module into the ICARAFE module, using the ICARAFE module to perform content-aware upsampling and high-fidelity detail reconstruction on each scale feature map, and performing fusion on the upsampled multi-scale features through a feature fusion network with a cross-scale connection structure to obtain a fused multi-scale defect feature map; S6, based on the fused multi-scale defect feature map, performing defect recognition and positioning through a detection head network, and outputting the category and position information of the steel surface defects.

2. The method of claim 1, wherein, The S1 comprises: S11, dividing the original image into fixed sizes, and extracting local image blocks in a sliding window manner, wherein the window size is set to N×N pixels, and an overlapping area of p% is reserved between adjacent windows, wherein p is a real number greater than 0; S12, sequentially performing brightness normalization, contrast stretching and Gaussian filter denoising on the divided image blocks; S13, applying an adaptive histogram equalization algorithm to enhance the local contrast of the image; S14, uniformly scaling the enhanced image to a standard input size; S15, performing data enhancement operations on the image, including random flipping, rotation, scaling, brightness disturbance and Gaussian noise injection; S16, dividing the processed image into a training set, a validation set and a test set in proportion.

3. The method of claim 1, wherein, The PentDConv module constructed in S2 realizes the generation of a dynamic convolution kernel through the following steps: S21, refining the global information of the input features to obtain global context representations in the channel dimension; S22, performing dimension reduction and nonlinear mapping on the global context representations through a feature compression layer; S23, based on a preset grouping number G dividing the input feature channel into G independent sub-channel groups, each sub-channel group containing C IN / G realizing the separation and interaction of local features and global features; C IN representing the number of channels of the input feature map S24, generate the attention weights of filter, space, channel and convolution kernel dimension respectively through four independent attention branches; wherein, the channel dimension attention weights are generated separately for each sub-channel group, for differentiating enhancing the sub-channel group dominated by defect features and suppressing the sub-channel group dominated by background texture. G sub-channel group, for differentiating enhancing the sub-channel group dominated by defect features and suppressing the sub-channel group dominated by background texture. S25, weighting and fusing the attention weights with predefined candidate convolution kernels to generate a dynamic convolution kernel adaptive to the input content.

4. The method of claim 1, wherein, The S3 comprises: S31, scaling and padding the preprocessed image to a preset size to form an input tensor; S32, performing preliminary feature extraction and down-sampling through a basic convolution layer; S33, inputting the feature map into the PentDConv module, and performing convolution operation using the generated dynamic convolution kernel to extract dynamic features; S34, the extracted features are sequentially passed through a C3 module composed of multiple bottleneck residual structures, a convolution downsampling layer, and an SPPF module to gradually extract deeper layer features.

5. The method of claim 1, wherein, The ICARAFE module constructed in S4 realizes content-aware upsampling through the following steps: S41, dimension reduction and coding of the input features are performed by a kernel prediction module to generate kernel prediction features; S42, the kernel prediction features are rearranged to an upsampling resolution using a PixelShuffle operation; S43, the rearranged features are normalized in the channel dimension by a Softmax function to obtain a spatially adaptive upsampling kernel tensor; S44, the input features are filled, local feature block extracted, and matrix multiplied and reorganized by a content-aware reorganization module using the upsampling kernel tensor to realize high-fidelity feature reconstruction.

6. The method of claim 1, wherein, S5 includes: S51, the multi-scale feature maps extracted by the PentDConv module are input into the ICARAFE module; S52, the upsampling features output by the ICARAFE module are fused through a feature fusion network with a cross-scale connection structure to fuse semantic and detail information at different levels.

7. The method of claim 1, wherein, S6 includes: S61, the fused multi-scale feature maps are input into three detection heads of different scales; the three detection heads process feature maps of different resolutions: 80x80x256, 40x40x512, and 20x20x1024, respectively, to optimize the detection of defects with a bounding box area less than 32x32 pixels, between 32x32 and 96x96 pixels, and greater than 96x96 pixels; S62, a set of reference boxes of different sizes are preset on each scale of feature map, and the predicted target position parameters are normalized and constrained; S63, the classification branch predicts the defect class probability, and the regression branch predicts the bounding box coordinates; S64, the classification probability and the target existence probability are fused to obtain the final detection confidence; S65, a composite loss function including classification loss, bounding box regression loss, and object loss is used for model training; S66, the prediction results are threshold filtered and non-maximum suppression (NMS) processed to remove redundant detection boxes, and the class label, confidence, and position coordinates of the defects are output. 8.The PCAR-YOLO based lightweight steel surface defect detection method of claim 2, wherein, The S1 standardization preprocessing of the collected steel surface image further includes: Performing lens distortion correction and brightness non-uniformity correction on the collected steel surface image to eliminate the influence of geometric distortion and uneven illumination introduced by the imaging system on the defect detection result; Based on the steel geometric profile and production process parameters, the region of interest (ROI) is determined, and the ROI of the corrected image is cropped to only retain the effective area containing the steel surface as the input for subsequent feature extraction.

9. A PCAR-YOLO-based steel surface defect lightweight detection device, characterized in that, It includes: An image acquisition and preprocessing module for performing the method steps S1 of any one of claims 1 to 8 to standardize and preprocess the collected steel surface image to form image data suitable for deep network analysis; a feature extraction module comprising a PentDConv module, configured to perform the method steps S2 and S3 of any one of claims 1 to 8, to receive the image data and perform multi-scale feature extraction to obtain enhanced defect feature representations; a feature reorganization and fusion module comprising an ICARAFE module and a feature fusion network, configured to perform the method steps S4 and S5 of any one of claims 1 to 8, to perform content-aware upsampling, high-fidelity detail reconstruction and cross-scale feature fusion on the enhanced defect feature representations to obtain fused multi-scale defect feature maps; a defect detection and output module, configured to perform the method step S6 of any one of claims 1 to 8, to perform defect recognition and localization based on the fused multi-scale defect feature maps and to output class and location information of surface defects of the steel.

10. The detection device of claim 9, wherein, The device is deployed on an industrial computer or an embedded AI computing platform and is communicatively connected with an industrial camera, a light source controller and a production line PLC to realize real-time online defect detection and sorting control on the surface of moving steel on a steel production line. The device is deployed on an industrial computer or an embedded AI computing platform and is communicatively connected with an industrial camera, a light source controller and a production line PLC to realize real-time online defect detection and sorting control on the surface of moving steel on a steel production line.

Images