Steel plate surface small defect detection method based on physical guidance and saliency enhancement

Abstract

This invention discloses a method for detecting small defects on steel plate surfaces based on physical guidance and saliency enhancement. Based on the Salience DETR model, a physical guidance saliency perception mechanism is proposed. The Retinex imaging physical prior is transformed into a learnable saliency guidance signal, which is embedded from the input end to the feature end, enhancing the ability to distinguish weak defects under complex lighting conditions. Simultaneously, a token sequence saliency score perception network is designed, using a feature-driven approach to replace traditional spatial distance calculation, and introducing adaptive modulation of the guidance signal to improve sensitivity to weak defects. Furthermore, a two-stage query refinement mechanism is constructed, combining the guidance signal to achieve dynamic offset optimization and multi-scale feature sampling of the query, promoting the refined recovery of small defect details. A local context adaptive fusion module is also designed, achieving feature semantic alignment through a dynamic cross-scale fusion strategy, alleviating the problem of small defect features being submerged during the fusion process.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement. It is suitable for the automated and intelligent detection of small defects on the surface of steel plates under complex lighting conditions, and can be widely used in industrial fields such as steel manufacturing, machining, and automobile manufacturing. Background Technology

[0002] As a crucial component of modern industrial systems, the surface quality of steel plates has a decisive impact on the safety and reliability of final products in the construction, machinery manufacturing, and even automotive industries. In complex production processes such as rolling, various microstructural anomalies inevitably form on the steel plate surface, including cracks, scratches, and pits. Traditional manual visual inspection methods are not only inefficient but also prone to visual fatigue due to their subjective judgment, making them unsuitable for meeting the dual requirements of high efficiency and high quality in current industrial production.

[0003] Of particular concern is the challenge of identifying minute defects during steel plate surface defect detection. When small defects with extremely low pixel counts are in complex lighting environments, their accurate identification rate drops significantly, becoming one of the main technical bottlenecks hindering the advancement of intelligent industrial quality inspection. Complex lighting environments specifically manifest as the degradation of steel plate surface image quality caused by the combined effects of strong reflections, uneven light distribution, and low image contrast. Examples show that the combined effects of fluctuating lighting conditions in production environments, the high reflectivity of the steel itself, and environmental shadow interference often lead to the characteristic information of minute defects being masked by background noise.

[0004] The primary challenge lies in the inherent weakness of the features of minute defects on the steel plate surface. In actual production scenarios, the defect sizes observed are typically extremely small, and the contrast difference between them and the background is insignificant, resulting in severely insufficient feature signal strength. Secondly, it's important to note the diversity of defect morphologies and their sparse spatial distribution: these defects often exhibit high similarity to natural textures, residual oil stains, or processing scratches on the steel surface. Therefore, the combined effect of weak features and complex backgrounds makes conventional detection algorithms perform poorly in small defect identification tasks—low-contrast areas are prone to missed detections, while high-noise areas frequently result in false positives.

[0005] Existing detection models also exhibit systemic deficiencies in their internal perception mechanisms and feature fusion methods.

[0006] Although deep learning-based defect detection methods have made progress, they still have three main shortcomings when facing industrial-level challenges: (1) Disconnect between physical perception and task reasoning: Most methods follow the “preprocessing-detection” serial paradigm, and the physical enhancement module is inconsistent with the detection task objective, resulting in the physical prior failing to be deeply integrated into the core perception process of the detector; (2) Coarse multi-scale feature fusion mechanism: Existing fusion strategies lack the ability to finely align and calibrate cross-scale semantics for the problem of small defect features being submerged and distorted due to semantic gaps; (3) Insufficient sensitivity to weak features of small defects: The feature selection or query initialization mechanism of existing models is slow to respond to extremely weak small defect features, and the filtering and refining process is difficult to effectively recover the fine details of small defects while suppressing background noise.

[0007] While the original Salience DETR model incorporates a saliency-guided query filtering mechanism, it lacks targeted recovery of details for small defects and fails to deeply integrate imaging physics priors into the detection process in a learnable and guided manner. Particularly under complex lighting conditions, the model's ability to distinguish weak defect features is limited, leading to a high rate of missed detections for small defects. Therefore, there is an urgent need to develop a steel plate surface defect detection technology that can deeply integrate physical perception and semantic understanding to improve the accuracy and robustness of small defect detection. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement, which aims to solve the problem of missed detection caused by the weak features of small defects on the surface of steel plates under complex lighting conditions and their susceptibility to being submerged by background information.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement includes: Step 1: Obtain an image of the steel plate surface and preprocess the image to obtain a standardized image; Step 2: Construct a benchmark detection model architecture based on Salience DETR, input standardized images into the backbone network, and extract multi-scale basic features; Step 3: Physically decompose and generate saliency-guided signals from the input image using a physically guided saliency perception mechanism. Step 4: The token sequence output by the encoder is predicted by feature-driven saliency score through the token sequence saliency score perception network, and adaptive fusion is performed in combination with the physical guidance signal to generate the final saliency score. Step 5: The query sequence is dynamically offset and multi-scale feature sampling is performed through a two-stage query refinement mechanism to achieve fine-grained restoration of small defect details; Step 6: Dynamically cross-scale semantic alignment and fusion of multi-scale features through the local context adaptive fusion module to alleviate the problem of small defect feature overwhelming; Step 7: Based on the prediction results output by the decoder, the Hungarian algorithm is used to match the query with the real target, the comprehensive loss function is obtained to optimize the model parameters, and the final detection result is output.

[0010] Furthermore, in step 1, the acquired steel plate surface image is preprocessed, including image resolution unification and pixel intensity normalization to ensure data distribution consistency; comprehensive data augmentation operations are used to improve the model's generalization ability, including random rotation, random horizontal / vertical flipping, brightness and contrast adjustment, and Gaussian noise addition; all processed images and their corresponding defect annotation information are stored in a standardized format to form standardized images.

[0011] Furthermore, in step 2, the baseline model architecture includes a backbone network and an encoder-decoder Transformer architecture. The backbone network uses the ResNet-50 architecture to extract multi-scale basic features of the image. The encoder part uses a multi-layer Transformer encoder layer to process multi-scale features and introduces position encoding to inject spatial position information, thereby enhancing the model's ability to perceive the spatial distribution of defects.

[0012] Furthermore, in step 3, the physically guided saliency perception mechanism includes color space conversion, multi-scale guided filter decomposition, adaptive gain enhancement, and guided modulation: Color space conversion transforms the input RGB image to the LAB color space, extracting the luminance component as the basis for physical decomposition. Multi-scale guided filtering decomposition uses a multi-scale guided kernel to perform edge-preserving smoothing on the luminance component, separating the smoothed illumination component from the detail reflection component. Adaptive gain enhancement dynamically adjusts the gain coefficient based on the local mean and standard deviation of the detail component, applying higher gain in low-contrast regions to highlight subtle defect features. Guided modulation generates a basic saliency map through Sigmoid function mapping and introduces dynamic guided intensity, noise suppression mask, and small target enhancement mask for fine modulation, generating the final physically guided saliency signal.

[0013] Furthermore, in step 4, the Token sequence saliency score-aware network includes bi-branch feature extraction, cross-channel attention mechanism, and adaptive guided fusion. The dual-branch feature extraction includes a global branch and a local branch, which respectively extract global semantic features through MLP and extract local correlation features through neighborhood aggregation; the cross-channel attention mechanism learns channel weights for global features to highlight key channel features; the adaptive guided fusion calculates uncertain weights based on model confidence and guided signal quality to achieve intelligent fusion of physical guided signals and model basic significance scores to generate the final significance score.

[0014] Furthermore, in step 5, the two-stage query refinement mechanism includes offset prediction and guided fusion, multi-scale adaptive sampling, and attention enhancement: Offset prediction and guidance fusion generate basic offsets based on query features and calculate guidance weights by combining physical guidance signals, dynamically fusing to generate the final sampled offset; multi-scale adaptive sampling dynamically predicts multi-scale weights for each query through a scale selection network to achieve adaptive sampling of multi-scale features; attention enhancement adopts a multi-head cross-attention mechanism to establish semantic associations between queries and sampled features, and strengthens semantic relationships between queries through efficient self-attention.

[0015] Furthermore, in step 6, the local context adaptive fusion module includes dual attention enhancement, dynamic cross-scale fusion strategy, and gated fusion: Dual attention enhancement enhances the discriminativeness of features at various scales through channel attention and spatial attention mechanisms; the dynamic cross-scale fusion strategy adaptively adjusts the fusion range and scale weights during training to achieve semantic alignment of multi-scale features; gating fusion achieves adaptive fusion of cross-scale features through gating weights and dynamic Dropout regularization, outputting semantically consistent multi-scale fused features.

[0016] Furthermore, in step 7, the Hungarian algorithm is used to match the query with the true defect bounding box. The matching criterion combines the predicted probability of the category with the intersection-union ratio of the bounding box. The comprehensive loss function includes classification loss, bounding box regression loss, and GIoU loss, as detailed below: The classification loss adopts an improved classification loss function based on focus loss, which balances easy and difficult samples through a dynamic weight adjustment mechanism, and is expressed as: Standard Sigmoid focus loss: ; ; in, For the model to the first The query predicts the first Sigmoid probabilities for each category For the corresponding category's real labels, The parameter for adjusting the weights of positive and negative samples. Adjust parameters for easy and difficult samples. Number of defect categories; Variant Sigmoid focus loss: ; ; For the first The intersection-union ratio of each predicted query box with the ground truth box is used as the loss contribution for confidence weighting to enhance high-quality positive samples. The bounding box regression loss uses L1 loss to measure the accuracy of bounding box coordinate prediction. ; in, and The first The predicted bounding box and ground truth bounding box coordinates for each matching query. The number of queries that were matched; GIoU loss measures the shape matching degree between the predicted bounding box and the ground truth bounding box: ; in, The generalized intersection-union-ratio (OCU) function is used for computation. The total loss function is the weighted sum of the losses of each component: ; in, , , These are the weight coefficients for classification loss, bounding box regression loss, and GIoU loss, respectively. The model parameters are optimized by minimizing the total loss function, and the final output is the detection results including defect category, bounding box coordinates, and confidence score.

[0017] Secondly, a steel plate surface defect detection system employing a physical guidance and saliency enhancement-based method for detecting small defects on steel plate surfaces includes an image acquisition module, a preprocessing module, a feature extraction and fusion module, a physical guidance module, a query optimization module, a detection output module, and a result post-processing module. The modules are seamlessly connected through feature channels.

[0018] Thirdly, an electronic device for industrial quality inspection scenarios includes a processor, a memory, an image acquisition device, a display device, and a data transmission module. These components are interconnected via a high-speed bus to form a highly efficient and collaborative hardware architecture, ensuring high-speed acquisition, real-time processing, and stable output of detection results for steel plate surface images. The memory pre-stores a computer-executable program, model training weight files, a defect category dictionary, and preset hyperparameter configurations. The processor loads and executes this computer program to accurately implement the aforementioned physical guidance and saliency enhancement-based method for detecting small defects on steel plate surfaces, completing the fully automated process from steel plate image input to accurate defect detection.

[0019] Fourthly, a computer-readable storage medium includes a computer program that, when executed by a processor, enables the above-described method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement, processing images of the steel plate surface and generating detection results.

[0020] The technical solution adopted in this invention has the following beneficial effects: This invention, based on the Salience DETR model, addresses the core issues of weak features, easy background obscuration, and multi-scale semantic inconsistencies in small defects on steel plate surfaces under complex lighting conditions. It constructs an end-to-end high-precision detection model by integrating a physically guided saliency perception mechanism, a token sequence saliency score perception network, a two-stage query refinement mechanism, and a local context adaptive fusion module. The overall process follows the logic of "physical perception embedding - feature saliency enhancement - query refinement recovery - cross-scale semantic alignment," sequentially completing image physically guided generation, multi-scale feature enhancement, dynamic query optimization, feature semantic alignment, and detection result calibration. Ultimately, it achieves accurate detection and reliable identification of small defects on steel plate surfaces in complex industrial environments. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the defect detection method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the defect detection method in an embodiment of the present invention; Figure 3 This is a schematic diagram of the TSSP-Net structure in an embodiment of the present invention; Figure 4 This is a schematic diagram of the TSQRM module structure in an embodiment of the present invention; Figure 5 This is a structural block diagram of the electronic device for detecting surface defects in steel plates according to the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] The flowchart of the method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement proposed in this invention is shown below. Figure 1 As shown, this method, based on the Salience DETR model, addresses the core issues of weak features, easy background obscuration, and multi-scale semantic inconsistency of small defects on steel plate surfaces under complex lighting conditions. It constructs an end-to-end high-precision detection model by integrating a physically guided saliency perception mechanism, a token sequence saliency score perception network, a two-stage query refinement mechanism, and a local context adaptive fusion module. The overall process follows the logic of "physical perception embedding - feature saliency enhancement - query refinement recovery - cross-scale semantic alignment - dynamic matching output," sequentially completing image physically guided generation, multi-scale feature enhancement, dynamic query optimization, feature semantic alignment, and detection result calibration. Ultimately, it achieves accurate detection and reliable identification of small defects on steel plate surfaces in complex industrial environments.

[0024] The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement in this embodiment of the invention is described in the following reference. Figure 1 As shown, it includes: The S100 preprocesses the acquired images. Standardized preprocessing is performed on the acquired steel plate surface images (covering surface images under different lighting and process conditions) to ensure the consistency and robustness of the model input. First, the image pixel intensity is normalized to the [0,1] interval to eliminate pixel distribution differences caused by uneven lighting. Then, all images are uniformly resampled to a standard size of 896×896 pixels using a bicubic interpolation algorithm, balancing detection accuracy and computational efficiency. To improve the model's generalization ability to complex industrial scenarios, a comprehensive data augmentation strategy is designed: random 90° rotation (50% trigger probability) and random horizontal / vertical flipping (50% trigger probability) are performed to simulate steel plate imaging scenarios under different lighting conditions and varying reflectivity. All preprocessed images and corresponding defect annotation information (including defect category and bounding box coordinates) are stored in the COCO standard format for model training and validation.

[0025] The S110 uses ResNet-50 as its backbone network and extracts multi-scale basic features from steel plate surface images layer by layer through convolution, pooling, and residual block stacking. The network consists of an initial convolutional layer (conv1), a max pooling layer, and four residual stages (layer1-layer4), outputting four feature maps at different scales. The downsampling factors relative to the input image are as follows: first-level feature map (layer1 output) 1 / 4 resolution, second-level feature map (layer2 output) 1 / 8 resolution, third-level feature map (layer3 output) 1 / 16 resolution, and fourth-level feature map (layer4 output) 1 / 32 resolution. The second to fourth level feature maps (1 / 8, 1 / 16, and 1 / 32 resolution) serve as the core feature layers, corresponding to shallow detail features, mid-level structural features, and deep semantic features, respectively, providing a multi-scale representation foundation for subsequent feature fusion and defect detection.

[0026] S120 inputs the preprocessed image into a Physically Guided Saliency Perception Mechanism (PGSPM) to generate a physically guided saliency signal related to defect semantics. First, the input RGB image is converted to the LAB color space, and the luminance component is extracted. Subsequently, multi-scale guided filtering was used to... Edge-preserving smoothing is performed to separate the smoothed illumination component B and the detail reflection component D. An adaptive gain enhancement strategy is applied to the detail component D, dynamically adjusting the gain coefficient based on the local mean and standard deviation. Higher gain is applied to low-contrast regions to highlight subtle defect features, while noise level estimation is used to suppress over-enhancement in high-noise regions. The enhanced detail component... The final physical guidance saliency signal is generated by using a Sigmoid function mapping as the basic saliency map and incorporating dynamic guidance strength, noise suppression mask, and small target enhancement mask for fine modulation. .

[0027] The multi-scale feature maps output from the S130 backbone network are input into the Transformer encoder for global context modeling, generating token sequences. The token sequence is then input into the Token Sequence Saliency Score Aware Network (TSSP-Net), the structure of which is as follows: Figure 3 As shown, the process is performed using a dual-branch feature extraction component: the global branch uses a multilayer perceptron (MLP) to learn a globally consistent semantic representation of the token. The local branch performs feature aggregation on the 5 neighboring tokens of each token to obtain basic local features. A cross-channel attention mechanism is introduced to weight and enhance global features, generating... The enhanced global features are concatenated with the local features to obtain the fused features. The baseline significance score is generated by the multi-level MLP prediction head. By combining the physical guidance signal G, an adaptive guidance fusion module calculates uncertain weights to achieve intelligent fusion of the guidance signal and the base score, generating the final significance score. .

[0028] S140 is based on significance scores. Perform stratified filtering on the queries to select highly significant query sequences. The query sequence is then input into a two-stage query refinement mechanism (TSQRM), the TSQRM structure of which is as follows: Figure 4 As shown, incremental optimization is performed. First, a basic sampling offset is generated based on the query features. And calculate the guidance intensity characteristics in combination with the physical guidance signal G. The confidence level of the query prediction is calculated using a confidence estimation network. By integrating query features and guidance strength features, the guidance credibility is output through a guidance quality assessment network. For the hard example regions with low confidence, a hard example enhancement coefficient γ=1.2 is introduced to increase the guiding weight, and the final sampling offset is dynamically fused. A scale selection network dynamically predicts the optimal scale weights for each query, achieving adaptive selection of multi-scale features. Based on Sparse sampling is performed on the feature maps at the corresponding scales to obtain a multi-scale sampled feature set. The multi-head cross-attention mechanism and efficient self-attention are combined to strengthen the semantic association between queries and output the optimized query features.

[0029] The S150 inputs multi-scale basic features into the Local Context Adaptive Fusion (LCASF) module for dynamic cross-scale semantic alignment fusion. First, it enhances the discriminative power of features at each scale through a channel-space dual attention mechanism, resulting in enhanced features. During training, the fusion range and scale weights are adaptively adjusted: the fusion scale range is linearly shrunk from the initial ±2 scales to ±1 scales; higher weights are assigned to high-resolution scales. Assigning moderate weight to low-resolution scales The features at the desired scale decay smoothly during training. After adjusting the features to be fused to the current scale, feature transformation, dynamic Dropout regularization, and gating weights are applied to achieve fusion, outputting semantically aligned multi-scale fused features. .

[0030] S160 performs object detection by combining the fused features with the optimized query input decoder. The Hungarian algorithm is used to match the query with the ground truth defect bounding boxes, with the matching criteria combining the predicted class probability and the intersection-union ratio (IU) of the bounding boxes. The comprehensive loss function includes a focus-based classification loss, a bounding box L1 regression loss, and a GIoU shape matching loss. The classification loss uses an improved focus loss function to address the imbalance between positive and negative samples for small defects on the steel plate surface. A dynamic weight adjustment mechanism balances the difficult and easy samples, including both standard and variant sigmoid focus loss implementations. The bounding box regression loss uses L1 loss to measure the accuracy of bounding box coordinate prediction. The GIoU loss measures the shape matching degree between the predicted and ground truth bounding boxes. The total loss function is a weighted sum of the various losses. The model parameters are optimized by minimizing the total loss function, ultimately outputting the defect category, bounding box coordinates, and confidence score. The detection results output by the decoder are post-processed and optimized: a non-maximum suppression algorithm (with an intersection-union threshold of 0.5) is used to remove duplicate detection boxes; a confidence threshold is applied to filter reliable detection results; the maximum number of detections per image is limited; and relative coordinates are converted to absolute image coordinates. The final output is a standardized detection result, including defect category, precise bounding box coordinates, and confidence score, supporting visualization and subsequent quality analysis.

[0031] The Hungarian algorithm is used to match the query with the true defect bounding box. The matching criterion combines the predicted probability of the category with the intersection-union ratio of the bounding box. The comprehensive loss function includes classification loss, bounding box regression loss, and GIoU loss, as detailed below: The classification loss adopts an improved classification loss function based on focus loss, which balances easy and difficult samples through a dynamic weight adjustment mechanism, and is expressed as: Standard Sigmoid focus loss: ; ; in, For the model to the first The query predicts the first Sigmoid probabilities for each category For the corresponding category's real labels, The parameter for adjusting the weights of positive and negative samples. Adjust parameters for easy and difficult samples. Number of defect categories; Variant Sigmoid focus loss: ; ; For the first The intersection-union ratio of each predicted query box with the ground truth box is used as the loss contribution for confidence weighting to enhance high-quality positive samples. The bounding box regression loss uses L1 loss to measure the accuracy of bounding box coordinate prediction. ; in, and The first The predicted bounding box and ground truth bounding box coordinates for each matching query. The number of queries that were matched; GIoU loss measures the shape matching degree between the predicted bounding box and the ground truth bounding box: ; in, The generalized intersection-union-ratio (OCU) function is used for computation. The total loss function is the weighted sum of the losses of each component: ; in, , , These are the weight coefficients for classification loss, bounding box regression loss, and GIoU loss, respectively. The model parameters are optimized by minimizing the total loss function, and the final output is the detection results including defect category, bounding box coordinates, and confidence score.

[0032] Furthermore, embodiments of the present invention also propose a steel plate surface defect detection system, such as... Figure 2 As shown, the system adopts an end-to-end architecture, mainly including an image acquisition module, a preprocessing module, a backbone network, a Physically Guided Saliency Perception Module (PGSPM), an encoder, a Token Sequence Saliency Score Perception Network (TSSP-Net), a Two-Stage Query Refinement Mechanism (TSQRM), a Local Context Adaptive Fusion Module (LCASF), a decoder, and a result post-processing module. Each module is seamlessly connected through a feature channel to ensure the continuity and effectiveness of feature transfer.

[0033] 1. Image Acquisition Continuous acquisition of images of steel plate surfaces on industrial production lines relies on industrial-grade CCD cameras or linear scanning imaging devices. Real-time transmission and high-resolution imaging capabilities ensure that the quality standards of the input images meet inspection requirements.

[0034] 2. Preprocessing module The standardization process encompasses image normalization, size specification standardization, and data augmentation techniques. This eliminates imaging variability and improves the model's generalization performance.

[0035] 3. The ResNet-50 backbone network architecture serves as the basic backbone network. Multi-scale feature extraction is accomplished through the residual connection mechanism, providing basic feature representation for subsequent processing stages.

[0036] 4. Physically Guided Saliency-Aware Component (PGSPM) The Retinex imaging theory forms its theoretical foundation. Learnable saliency-guided signals are derived from physical priors, and physical perception is embedded in the process from input to feature. This enhances the discriminative power of subtle defect features under complex lighting conditions.

[0037] 5. Encoder Structure The global context modeling design is performed on multi-scale features by a multi-layer Transformer encoder. Token sequences containing rich semantic information are thus generated.

[0038] 6. Token Sequence Saliency Score Aware Network (TSSP-Net) Abandoning the traditional spatial distance calculation model, a novel feature-driven calculation method is adopted. By integrating physical guidance signals to complete adaptive modulation, the sensitivity to the perception of weak defect features is improved, and finally, a high-confidence saliency score is generated, providing accurate guidance for subsequent small defect identification.

[0039] 7. Two-Stage Query Refinement Mechanism (TSQRM) A progressive and refined recovery strategy is implemented for the query sequence. Through the synergy of dynamic offset optimization and multi-scale feature sampling, the model accurately captures the details of small defects, greatly enhances the model's ability to perceive the details of small defects, and avoids the loss of details in the query filtering and refining process.

[0040] 8. Local Context Adaptive Fusion Component (LCASF) A dynamic cross-scale fusion strategy is adopted to achieve accurate semantic alignment between features at multiple scales, thereby alleviating the problem of deep strong semantic features drowning out shallow small defect weak features during the multi-scale fusion process and ensuring the effective transmission of small defect features.

[0041] 9. Decoder Based on optimized query features and fused features, end-to-end ensemble prediction is performed, and defect category determination and bounding box coordinate output are completed simultaneously to ensure the real-time and consistency of detection results, meeting the high-efficiency requirements of industrial online inspection.

[0042] 10. Result Post-processing Module It integrates three core functions: Non-maximum suppression (NMS), confidence threshold filtering, and bounding box coordinate transformation. The NMS algorithm has a cross-union threshold of 0.5, and the confidence filtering threshold is set to 0.05. Through three steps—eliminating duplicate detection boxes, filtering high-confidence valid results, and transforming the coordinate system—it comprehensively ensures the accuracy, recall, and positioning precision of the final detection results, fully meeting the high standards required for industrial quality inspection.

[0043] The various modules work together to form a complete intelligent inspection system, which significantly improves the detection accuracy and robustness of small defects on the surface of steel plates, providing a highly reliable technical solution for the field of industrial quality inspection.

[0044] III. Core Technological Advantages This invention achieves four core technological breakthroughs through multi-module collaborative optimization design, effectively overcoming the pain points of existing steel plate small defect detection technologies: First, the PGSPM module innovatively transforms Retinex imaging physical priors into learnable saliency guidance signals, breaking the industry bottleneck of the disconnect between physical perception and task reasoning under the traditional "preprocessing-detection" serial paradigm. This achieves end-to-end deep fusion of physical perception and semantic understanding, significantly improving the model's ability to distinguish weak defect features under complex lighting conditions and solving the problem of difficult identification of small defects under complex lighting. Second, TSSP-Net replaces the traditional spatial distance calculation scheme with a feature-driven approach, combining physical guidance signals to complete adaptive fusion modulation. This overcomes the shortcomings of traditional query selection mechanisms being insensitive to weak features, while enhancing the model's response to small defects and robustness against annotation errors, improving the stability of detection results. Third, TSQRM... By combining physical guidance signals, dynamic offset optimization and multi-scale feature sampling are implemented on the query sequence, which effectively avoids the loss of small defect details in the query filtering and refining process, promotes the fine recovery of small defect details, and improves the accuracy of small defect detection. Fourth, LCASF adopts a dynamic cross-scale fusion strategy to achieve multi-scale feature semantic alignment, solves the common problem of deep strong semantic features submerging shallow weak features in multi-scale feature fusion, effectively alleviates the information submersion problem of small defect features in the fusion process, and ensures the effective utilization of small defect features.

[0045] The aforementioned technological innovations create a synergistic effect, comprehensively improving the model's detection accuracy, recall rate, and scenario adaptability for small defects in complex industrial scenarios, providing more efficient and reliable technical support for industrial steel plate quality inspection.

[0046] To verify the feasibility and effectiveness of the embodiments of the present invention, specific embodiments were selected and compared with existing mainstream detection methods. Table 1 shows the detailed experimental results of the method of the present invention on the NEU-DET steel plate surface defect detection dataset. The experiments selected average accuracy (AP) and small target average accuracy (AP) as the experimental parameters. S ) and average recall rate (AR) S () is used as the core evaluation indicator to comprehensively measure model performance.

[0047] Table 1. Comprehensive comparison results with advanced detectors

[0048] Both model training and validation in this experiment were performed on an NVIDIA RTX 4090 GPU (24GB VRAM). The model was built and optimized based on the PyTorch 2.1.0 framework, and computational acceleration was achieved using CUDA 12.1. All hyperparameters used in the experiment were optimized and determined: the batch size was set to 4, and the initial learning rate was 1×10⁻⁶. -4 The learning rate is updated using a cosine annealing scheduling strategy. The model on the NEU-DET dataset is trained for a total of 70 epochs, and the weight decay coefficient is configured to be 0.0001.

[0049] In the experiment, the NEU-DET dataset images were first preprocessed using S100 normalization to adjust all images to the model input size, and then multiple data augmentation methods were performed. Next, the ResNet-50 backbone network extracted multi-scale basic features, and the core Physically Guided Satisfaction Perception Module (PGSPM) simultaneously generated guiding signals associated with defect semantics, providing physical perception prior information for subsequent feature augmentation and query optimization.

[0050] After processing by the Token Sequence Saliency Score Aware Network (TSSP-Net), the model calculates saliency scores in a feature-driven manner and completes adaptive modulation in conjunction with the aforementioned guiding signal, effectively improving the sensitivity to small defects. The two-stage query refinement mechanism (TSQRM) achieves refined recovery of small defect details through dynamic offset optimization and multi-scale feature sampling. The Local Context Adaptive Fusion Module (LCASF) relies on a dynamic cross-scale fusion strategy to effectively alleviate the information overload problem of small defect features in the multi-scale fusion process.

[0051] The model ultimately outputs the detection results of surface defects in the steel plate. The experiment uses the average accuracy (AP) and the average accuracy of small targets (AP) as the benchmarks. S The core evaluation metric is . Experimental data shows that this method achieves an AP of 0.439 and an AP of 0.347 on the NEU-DET dataset. SWith an AR value of 0.493 S All indicators are superior to existing comparative methods, and the performance is particularly outstanding in small defect detection tasks, which fully verifies the significant advantages of this method in physical perception embedding, weak defect feature enhancement and fine-grained recovery of small defect details.

[0052] In this embodiment of the invention, a structural diagram of an electronic device for detecting surface defects in steel plates for industrial quality inspection is provided. Figure 5 As shown.

[0053] This electronic device integrates a processor, memory, input devices (such as industrial CCD cameras and line scan scanners), and output devices (such as industrial displays, alarm modules, and data interfaces) via a high-speed system bus. Image data of the steel plate surface acquired by the input devices is transmitted via the system bus to the memory for storage and the processor for computation. After the processor executes the defect detection algorithm and performs data analysis, it transmits the detection results (including defect category, bounding box, and confidence level) to the output devices via the bus. The results are then presented visually, trigger alarms, or uploaded to the production management system via the data interface, providing accurate data support for real-time quality monitoring, defect classification statistics, and process optimization.

[0054] The processor is the core of the equipment, responsible for loading and executing the physical guidance and saliency enhancement-based small defect detection method for steel plate surfaces stored in memory. It integrates a multi-core CPU and a high-performance GPU, efficiently running core algorithms such as the physical guidance saliency perception module, the token sequence saliency score perception network, the two-stage query refinement mechanism, and the local context adaptive fusion module through a parallel computing architecture. This allows for multi-scale feature extraction, physical perception embedding, dynamic query optimization, and precise defect localization of the input steel plate image, rapidly outputting high-confidence defect detection results to meet the real-time requirements of high-speed continuous inspection on the production line.

[0055] The memory includes ROM and RAM, and is expanded with a high-speed solid-state storage array. ROM is used to store the system program, defect detection algorithm kernel and preset parameters (such as defect category dictionary and model hyperparameters); RAM serves as a high-speed cache to store intermediate data of image preprocessing, multi-scale feature maps and temporary data of the decoding process; the solid-state storage array is used for long-term storage of historical detection images, model weight files and detection result logs, supporting data traceability and model iterative updates.

[0056] The input device is centered on an industrial image acquisition unit, equipped with a high-resolution area array CCD camera or a high-speed line array scanning camera, and a uniform light source system to ensure the acquisition of high-quality images of the steel plate surface under complex lighting conditions. It also supports the import of stored image datasets via a network interface.

[0057] The output devices include an industrial display, an audible and visual alarm module, and a data interface module: the industrial display can visualize the inspection results in real time, mark the defect area with different colored bounding boxes, and simultaneously display the type, size, and confidence level of the defect; the audible and visual alarm module will automatically start when a serious defect is detected; the data interface module can export the inspection results in association with the original image, and can also be directly uploaded to the Manufacturing Execution System (MES) and Quality Management System (QMS).

[0058] Based on the above embodiments, the electronic equipment involved in this invention can directly run the method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement, and is suitable for various industrial quality inspection scenarios such as online inspection of continuous production lines and offline sampling inspection, and has the advantages of high real-time performance, high robustness and strong compatibility.

[0059] This invention can be implemented in various forms, including methods, systems, and computer program products. Specifically, it can be achieved in three ways: pure software deployment, where the detection algorithm is integrated into an industrial control computer or server; pure hardware integration, where the core functions are integrated into a dedicated vision inspection device; and a combination of software and hardware, where embedded GPU devices are paired with customized inspection software, allowing for flexible adaptation to different production line automation levels and budget requirements. Furthermore, this invention can also be packaged as a computer program product, with its program code stored on computer-readable storage media such as solid-state drives, industrial CF cards, and cloud servers, supporting rapid deployment and version upgrades across different inspection devices.

[0060] The specific implementation process of this invention can be referred to the relevant flowcharts. The diagrams clearly illustrate the operating logic and algorithm execution steps of the electronic device. Each functional module is implemented through a series of computer program instructions. These instructions can be transmitted to the processing unit of a general-purpose industrial computer, a dedicated vision controller, or an embedded system. The device will execute the instructions according to the predetermined logic to complete the entire process of steel plate image acquisition, processing, defect detection, and result output. These instructions can also be stored in a computer-readable storage medium. When the device is loaded and executed, it guides the device to complete preset functions or allows for flexible on-site adjustment of detection parameters through instruction combinations.

[0061] Although the main embodiments of the present invention have been described, those skilled in the art, after understanding the core innovative ideas, can make reasonable modifications or adjustments without departing from the technical framework (such as adapting to surface detection of different materials, expanding the defect category library, and optimizing the real-time processing pipeline). Therefore, the appended claims should cover all such modifications and changes, provided that they fall within the protection scope defined by the present invention.

Claims

1. A method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement, characterized in that, include: Step 1: Obtain an image of the steel plate surface and preprocess the image to obtain a standardized image; Step 2: Construct a benchmark detection model architecture based on Salience DETR, input standardized images into the backbone network, and extract multi-scale basic features; Step 3: Physically decompose and generate saliency-guided signals from the input image using a physically guided saliency perception mechanism. Step 4: The token sequence output by the encoder is predicted by feature-driven saliency score through the token sequence saliency score perception network, and adaptive fusion is performed in combination with the physical guidance signal to generate the final saliency score. Step 5: The query sequence is dynamically offset and multi-scale feature sampling is performed through a two-stage query refinement mechanism to achieve fine-grained restoration of small defect details; Step 6: Dynamically cross-scale semantic alignment and fusion of multi-scale features through the local context adaptive fusion module to alleviate the problem of small defect feature overwhelming; Step 7: Based on the prediction results output by the decoder, the Hungarian algorithm is used to match the query with the real target, the comprehensive loss function is obtained to optimize the model parameters, and the final detection result is output.

2. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 1, the obtained steel plate surface image is preprocessed, including image resolution unification and pixel intensity normalization, to ensure the consistency of data distribution. Comprehensive data augmentation operations are employed to enhance the model's generalization ability. These operations include random rotation, random horizontal / vertical flipping, brightness and contrast adjustment, and Gaussian noise addition. All processed images and their corresponding defect annotation information are stored in a standardized format to form standardized images.

3. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 2, the baseline model architecture includes a backbone network and an encoder-decoder Transformer architecture. The backbone network uses the ResNet-50 architecture to extract multi-scale basic features of the image. The encoder part uses a multi-layer Transformer encoder layer to process multi-scale features and introduces position encoding to inject spatial position information, thereby enhancing the model's ability to perceive the spatial distribution of defects.

4. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 3, the physically guided saliency perception mechanism includes color space conversion, multi-scale guided filter decomposition, adaptive gain enhancement, and guided modulation: Color space conversion converts the input RGB image to the LAB color space and extracts the luminance component as the basis for physical decomposition; multi-scale guided filtering decomposition uses a multi-scale guided kernel to perform edge-preserving smoothing on the luminance component, separating the smoothed illumination component and the detail reflection component; adaptive gain enhancement dynamically adjusts the gain coefficient based on the local mean and standard deviation of the detail component, applying higher gain in low-contrast areas to highlight subtle defect features. The guided modulation generates a basic saliency map through a Sigmoid function mapping, and then introduces dynamic guidance strength, noise suppression mask and small target enhancement mask for fine modulation to generate the final physically guided saliency signal.

5. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 4, the Token sequence saliency score-aware network includes bi-branch feature extraction, cross-channel attention mechanism, and adaptive guided fusion. Dual-branch feature extraction includes a global branch and a local branch, which respectively extract global semantic features through MLP and extract local correlation features through neighborhood aggregation; The cross-channel attention mechanism learns channel weights for global features, highlighting key channel features; Adaptive guided fusion calculates uncertain weights based on model confidence and guided signal quality, enabling intelligent fusion of physical guided signals and model basic significance scores to generate the final significance score.

6. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 5, the two-stage query refinement mechanism includes offset prediction and guided fusion, multi-scale adaptive sampling, and attention enhancement: Offset prediction and guidance fusion generate basic offsets based on query features and calculate guidance weights by combining them with physical guidance signals, and dynamically fuse them to generate the final sampled offsets; multi-scale adaptive sampling dynamically predicts multi-scale weights for each query through a scale selection network to achieve adaptive sampling of multi-scale features; Attention enhancement employs a multi-head cross-attention mechanism to establish semantic associations between queries and sampled features, and strengthens semantic relationships between queries through efficient self-attention.

7. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 6, the local context adaptive fusion module includes dual attention enhancement, dynamic cross-scale fusion strategy, and gated fusion: Dual attention enhancement enhances the discriminativeness of features at various scales through channel attention and spatial attention mechanisms; the dynamic cross-scale fusion strategy adaptively adjusts the fusion range and scale weights during training to achieve semantic alignment of multi-scale features; gating fusion achieves adaptive fusion of cross-scale features through gating weights and dynamic Dropout regularization, outputting semantically consistent multi-scale fused features.

8. The method for detecting small defects on the surface of steel plates based on physical guidance and saliency enhancement according to claim 1, characterized in that, In step 7, the Hungarian algorithm is used to match the query with the true defect bounding box. The matching criterion combines the predicted probability of the category with the intersection-union ratio of the bounding box. The comprehensive loss function includes classification loss, bounding box regression loss, and GIoU loss, as follows: The classification loss adopts an improved classification loss function based on focus loss, which balances easy and difficult samples through a dynamic weight adjustment mechanism, and is expressed as: Standard Sigmoid focus loss: ； ； in, For the model to the first The query predicts the first Sigmoid probabilities for each category For the corresponding category's real labels, The parameter for adjusting the weights of positive and negative samples. Adjust parameters for easy and difficult samples. Number of defect categories; Variant Sigmoid focus loss: ； ； For the first The intersection-union ratio of each predicted query box with the ground truth box is used as the loss contribution for confidence weighting to enhance high-quality positive samples. The bounding box regression loss uses L1 loss to measure the accuracy of bounding box coordinate prediction. ； in, and The first The predicted bounding box and ground truth bounding box coordinates for each matching query. The number of queries that were matched; GIoU loss measures the shape matching degree between the predicted bounding box and the ground truth bounding box: ； in, The generalized intersection-union-ratio (OCU) function is used for computation. The total loss function is the weighted sum of the losses of each component: ； in, , , These are the weight coefficients for classification loss, bounding box regression loss, and GIoU loss, respectively. The model parameters are optimized by minimizing the total loss function, and the final output is the detection results including defect category, bounding box coordinates, and confidence score.

9. A steel plate surface defect detection system employing the physical guidance and saliency enhancement-based small defect detection method for steel plate surfaces as described in claim 1, characterized in that, It includes an image acquisition module, a preprocessing module, a feature extraction and fusion module, a physical guidance module, a query optimization module, a detection output module, and a result post-processing module. The modules are seamlessly connected through feature channels.

Images