Material surface micro-defect recognition method based on deep learning

Abstract

This application relates to a deep learning-based method for identifying micro-defects on material surfaces. The method includes: performing image enhancement processing on a material surface image to obtain an enhanced micro-defect image; inputting the enhanced micro-defect image into a resolution-preserving multi-scale feature extraction network, extracting multi-scale feature maps while maintaining the spatial resolution of the input image, and performing attention-weighted processing to obtain an attention-enhanced feature map; performing dual-domain attention fusion processing on the attention-enhanced feature map to obtain a fused feature map; performing micro-defect segmentation based on the fused feature map to obtain a micro-defect segmentation mask, and performing post-processing on the micro-defect segmentation mask. The output micro-defect identification result includes defect category, defect location coordinates, and defect contour information. This application avoids the loss of micro-defects during downsampling by preserving resolution feature extraction and enhances the representational ability of micro-defect features by combining spatial and frequency domain cross-attention fusion, thereby improving the identification accuracy and localization accuracy of micro-defects.

Description

Technical Field

[0001] This application relates to the field of surface treatment, and in particular to a method for identifying micro-defects on material surfaces based on deep learning. Background Technology

[0002] In industrial manufacturing, the surface quality of materials directly affects the reliability and lifespan of products. The surfaces of precision products such as metal sheets, semiconductor wafers, and optical components often contain micro-defects ranging from micrometers to sub-millimeters, such as scratches, pitting, cracks, and inclusions. Although these micro-defects are small, they can lead to stress concentration, electrical short circuits, or deterioration of optical performance. Traditional manual visual inspection methods are slow, lack standardized procedures, and are prone to missing defects, making them unable to meet the dual requirements of modern production lines for both speed and accuracy.

[0003] With the development of deep learning technology in image recognition, defect detection methods based on convolutional neural networks have been gradually applied to industrial surface inspection. However, existing deep learning detection methods face a core contradiction throughout the entire detection process when dealing with micro-defects on material surfaces: the conflict between the weakness of the micro-defect signal and the complexity of the background texture of the material surface. Micro-defects manifest as low-contrast, small-area localized gray-level anomalies in images, and their signal intensity is often on the same order of magnitude as the periodic texture variations of the material surface itself. This makes it difficult to effectively separate the micro-defect signal from the background texture noise during the image acquisition stage.

[0004] This core contradiction triggers a chain reaction of problems at every stage of the detection process. In the feature extraction stage, mainstream multi-scale detection networks rely on a feature pyramid structure with progressive downsampling to expand the receptive field. However, the spatial information of micro-defects decays layer by layer during downsampling. When the feature map resolution drops to a fraction of the original image, micro-defect features occupying only a few pixels may be completely discarded. Although upsampling and skip connections can partially recover spatial information, the recovered features are already mixed with positional offsets and interpolation artifacts introduced in the low-resolution stage, making it impossible to restore the precise boundaries of micro-defects. Therefore, the traditional downsampling pyramid structure is insufficient in preserving the features of micro-defects.

[0005] In the feature fusion stage, existing methods typically employ spatial domain attention mechanisms to enhance the feature representation of defect regions. However, the spatial domain features of micro-defects are often obscured by background texture, making it difficult to distinguish between the two through attention operations only in the spatial domain. The periodic background texture of a material surface manifests as concentrated frequency peaks in the frequency domain, while the frequency domain features of micro-defects are dispersed across different frequency bands and lack significant directional regularity. If frequency domain information is ignored during the feature fusion stage, and feature selection is performed solely based on spatial domain attention, the separability of defect signals and background textures in the frequency domain cannot be utilized, resulting in limited representational ability of the fused features for micro-defects.

[0006] In the defect identification stage, the reliability of micro-defect segmentation results is directly affected by the aforementioned insufficient feature quality. When the segmentation network is uncertain about the defect attribution of a certain region, the fixed threshold method can only make a compromise between missed detections and false detections. In addition, different types of micro-defects, such as linear scratches and diffuse corrosion, have different sensitivities to post-processing constraints, and a uniform post-processing rule cannot simultaneously meet the screening requirements of multiple defect types.

[0007] Therefore, a material surface micro-defect identification method is needed that can collaboratively address the contradiction between the weakness of micro-defect signals and the complexity of background textures throughout the entire process from image enhancement, feature extraction, feature fusion to defect determination. Summary of the Invention

[0008] To overcome the contradiction between the weakness of micro-defect signals and the complexity of background texture, and to improve the recognition accuracy and localization accuracy of micro-defects, this application provides a deep learning-based method for identifying micro-defects on material surfaces.

[0009] This application provides a deep learning-based method for identifying micro-defects on material surfaces, employing the following technical solution: A deep learning-based method for identifying micro-defects on material surfaces includes the following steps: S1. Acquire material surface images, perform image enhancement processing on the acquired material surface images, and obtain enhanced images of micro-defects; S2. Input the micro-defect enhanced image into the resolution-preserving multi-scale feature extraction network, extract multi-scale feature maps while maintaining the spatial resolution of the input image, and perform attention weighting processing on the multi-scale feature maps to obtain attention-enhanced feature maps; S3. Perform dual-domain attention fusion processing on the attention-enhanced feature map to obtain the fused feature map; S4. Perform micro-defect segmentation based on the fused feature map, obtain the micro-defect segmentation mask, perform post-processing steps on the micro-defect segmentation mask, and output the micro-defect recognition result, which includes the defect category, defect location coordinates, and defect contour information.

[0010] By adopting the above technical solutions, the resolution-preserving multi-scale feature extraction network in S2 extracts multi-scale feature maps while maintaining the spatial resolution of the input image. This ensures that the spatial details of micro-defects, which occupy only a few pixels in the original image, are not attenuated or lost due to downsampling during the entire feature extraction process. Each scale feature map retains the same spatial localization accuracy as the original image, thus providing pixel-level accurate defect location information for subsequent segmentation steps. In S3, the dual-domain attention fusion processing cross-enhances the attention information in both the spatial and frequency domains. This allows the fused feature map to simultaneously capture the spatial distribution features of micro-defects in the spatial domain and the spectral features that distinguish it from the background texture in the frequency domain. The attention information from the two domains complement each other. Spatial domain attention guides frequency domain features to focus on the spatial region where the defect is located, while frequency domain attention guides spatial domain features to focus on the unique spectral channels of the defect. The cross-action of the two enhances the ability of the fused features to distinguish micro-defects from the background texture compared to the single-domain attention scheme. In S4, the micro-defect segmentation mask is post-processed and the defect category, location coordinates, and contour information are output. This allows the identification results to not only indicate the presence or absence of a defect, but also provide precise spatial location and quantitative description of the defect's shape and contour, meeting the needs of industrial inspection for defect classification, statistics, and size assessment.

[0011] Optionally, S1 includes the following sub-steps: S11. Obtain a material surface image, perform a two-dimensional fast Fourier transform on the material surface image, and obtain a frequency domain feature map; S12. Radial integration is performed on the amplitude spectrum of the frequency domain feature map to obtain the radial power spectral density distribution. Peak detection is performed on the radial power spectral density distribution to obtain at least one main peak frequency. Each main peak frequency corresponds to the spatial frequency of the periodic background texture in the material surface image. In response to the fact that there are multiple main peak frequencies obtained by peak detection, a cascaded notch filter bank is used to filter the frequency domain feature map. The cascaded notch filter bank includes multiple directional notch filters that correspond one-to-one with the multiple main peak frequencies. The notch region of each directional notch filter is a fan-shaped region with the corresponding main peak frequency as the radius and the corresponding main direction of the background texture as the center. Each directional notch filter only suppresses the frequency component in the corresponding main direction of the background texture and retains the frequency component that is the same as the corresponding main peak frequency but has a different direction. Multiple directional notch filters are cascaded on the frequency domain feature map in descending order of the amplitude value of the corresponding main peak frequency in the radial power spectral density distribution to obtain the filtered frequency domain feature map. S13. Perform a two-dimensional inverse fast Fourier transform on the filtered frequency domain feature map to obtain the frequency domain enhanced image; S14. Perform local adaptive contrast enhancement processing on the frequency domain enhanced image to obtain a micro-defect enhanced image.

[0012] By employing the above technical solutions, peak detection of radial power spectral density distribution can automatically identify the spatial frequencies corresponding to each periodic background texture in a material surface image, adapting to the texture characteristics of different materials without the need for manual preset filtering parameters. A cascaded notch filter bank suppresses multiple main peak frequencies one by one, acting sequentially in descending order of amplitude value. This ensures that the strongest background texture component is preferentially eliminated, preventing its sidelobes from obscuring the detection of weaker texture components. The notch region of each directional notch filter is a fan-shaped area with the main peak frequency as its radius and the main texture direction as its center. It suppresses only the frequency component in the main texture direction while retaining components of the same frequency but different directions. This avoids mistakenly suppressing micro-defect frequency components with frequencies similar to the background texture but different directions, preserving the integrity of the micro-defect signal while eliminating background texture interference. Local adaptive contrast enhancement is performed on the frequency-domain filtered image. Since the energy of the background texture has been suppressed, the enhancement operation primarily amplifies the micro-defect signal rather than background variations, thus improving the visibility of micro-defects.

[0013] Optionally, an image preprocessing step may be included before performing S1: S01. Obtain a background baseline model constructed from a set of defect-free images of the same material and batch; S02. Perform a difference operation between the material surface image and the background baseline model to obtain the residual image; S03. Replace the material surface image with the residual image as the input of S1; In step S1, the image enhancement process uses a multi-channel adaptive filter bank to filter the frequency domain feature map of the residual image. The multi-channel adaptive filter bank includes multiple parallel narrowband filters, each covering a different frequency band. The outputs of each narrowband filter are weighted and combined using learnable channel weights to obtain the filtered frequency domain feature map.

[0014] By adopting the above technical solution, the background baseline model is constructed from defect-free images of the same material and batch. Differential operations eliminate the background texture components in the material surface image that are consistent with the background baseline. The residual image mainly retains the defect signal and a small number of residual components caused by differences in acquisition conditions. The subsequent multi-channel adaptive filter bank performs frequency domain filtering on the residual image instead of the original image. Since the background texture energy of the residual image has been eliminated by differential operations, each narrowband filter operates on a signal with a higher signal-to-noise ratio. The learnable channel weights can more accurately learn the response patterns of different types of micro-defects in different frequency bands, so that differential operations and frequency domain filtering are optimized synergistically during training. The combined effect of the two is better than the sum of the effects when they are applied independently.

[0015] Optionally, in S2, the resolution-preserving multi-scale feature extraction network includes a basic convolutional layer, a parallel dilated convolutional group, and an edge-aware attention module; wherein, the basic convolutional layer is used to perform convolution operations on the micro-defect enhancement image to obtain a basic feature map, and the spatial resolution of the basic feature map is the same as that of the micro-defect enhancement image; the parallel dilated convolutional group includes multiple parallel dilated convolutional branches, and the output feature map of each dilated convolutional branch maintains the same spatial resolution as the basic feature map; The edge-aware attention module is used to perform learnable edge detection convolution operations on the basic feature map to obtain an edge probability map, perform diffusion processing on the edge probability map to obtain an edge attention map, and apply the edge attention map to the output feature map of the parallel dilated convolution group to obtain an attention-enhanced feature map. The diffusion radius of the diffusion process is a pixel-level adaptive parameter. The diffusion radius of each pixel is determined based on the edge intensity value of the corresponding pixel in the edge probability map. The diffusion radius and the edge intensity value are inversely proportional.

[0016] By employing the above technical solution, each branch of the parallel dilated convolutional group acquires features with different receptive field ranges at the same spatial resolution using different dilation rates. This allows the network to simultaneously perceive the local details of micro-defects and their surrounding contextual information, unlike traditional downsampling schemes which sacrifice spatial resolution for a larger receptive field. The edge-aware attention module generates an edge probability map through learnable edge detection convolutional operations, concentrating attention weights on the edge region of the micro-defect and its neighboring regions, suppressing feature responses from large areas of normal background far from the defect edge. The diffusion radius is inversely proportional to the edge intensity value, allowing micro-defects with high edge intensity to obtain a small diffusion radius for precise spatial focusing, while large-scale defect boundaries with low edge intensity obtain a large diffusion radius to cover the width of the defect. Micro-defects of different scales each obtain an attention coverage range that matches their morphology.

[0017] Optionally, before the determination of the diffusion radius is driven by the edge probability map, the following processing is further included: calculating the edge confidence index of each pixel position based on the concentration of the gray-level gradient direction distribution of each pixel in the neighborhood of each pixel position in the basic feature map; in response to the edge confidence index of the pixel position being higher than the gate threshold, transferring the edge probability value of the corresponding pixel position in the edge probability map to the diffusion radius determination stage; in response to the edge confidence index of the pixel position not being higher than the gate threshold, setting the edge probability value of the corresponding pixel position in the edge probability map to zero.

[0018] By employing the above technical solution, the gray-level gradient direction distribution of each pixel at the edge of a real micro-defect is highly concentrated because the defect boundary has a consistent orientation; while the gray-level gradient direction distribution of each pixel at pseudo-edges generated by residual material surface texture or acquisition noise is randomly dispersed and lacks consistent directionality. The edge confidence index utilizes this physical difference to filter pseudo-edges from the edge probability map. Pseudo-edge pixels that are zeroed out no longer participate in the calculation of the diffusion radius, thus avoiding the problem of noisy edges obtaining inappropriately large diffusion radii, including non-defect areas within the attention range, and consequently triggering positive feedback noise amplification.

[0019] Optionally, the attention weighting processing in S2 employs a bimodal attention mechanism, which includes an edge attention modality, a region attention modality, and a modality selection classifier based on the edge-aware attention module. The attention map of the edge attention modality is generated through learnable edge detection convolution operations and diffusion processing; The attention map of the region attention mode is generated by the region response operation with a large convolution kernel. The region attention mode is used to perform attention weighting on area-type defect regions with blurred boundaries. The modality selection classifier is used to output continuous modality mixing weights based on the local features of each spatial location in the attention enhancement feature map. The modality mixing weights include edge modality weights and region modality weights. The attention map of the edge attention modality and the attention map of the region attention modality are weighted and superimposed according to the modality mixing weights to obtain a mixed attention map. The modal mixing weights are passed down to the post-processing stage of S4 to drive the determination of constraint parameters in the post-processing.

[0020] By employing the above technical solutions, the edge attention modality focuses on linear defect regions with clear boundaries through edge detection convolution operations and diffusion processing, while the region attention modality focuses on area-type defect regions with blurred boundaries through region response operations with large convolutional kernels. The two modalities apply attention weights to two typical micro-defect morphologies respectively. The modality selection classifier outputs continuous modality mixing weights instead of binary selection, enabling the attention maps of the two modalities to be proportionally superimposed and synergistically applied to mixed regions containing both linear and area-type defects, rather than activating only one modality in an either-or manner. The modality mixing weights are passed down to the post-processing stage S4, allowing the defect morphology judgment in the feature extraction stage to directly drive the constraint parameter selection in the post-processing stage. The front-end and back-end work collaboratively through the same parameter signal, eliminating the need for independent re-judgment of defect types in the post-processing stage.

[0021] Optionally, step S3 includes the following sub-steps: S31. Perform global average pooling and global max pooling on the attention enhancement feature map along the channel dimension respectively. After concatenating the pooling results, generate a spatial attention weight map through a convolutional layer. Multiply the spatial attention weight map with the attention enhancement feature map element by element to obtain the spatial attention features. S32. Perform a two-dimensional discrete cosine transform on each channel of the attention enhancement feature map, input the transform domain coefficients into the channel attention network to generate a frequency attention weight map, and multiply the frequency attention weight map with the attention enhancement feature map element by element to obtain the frequency attention features. S33. Weight the frequency attention features with the spatial attention weight map, and weight the spatial attention features with the frequency attention weight map. Add the two weighted results to obtain the fused feature map.

[0022] By employing the above technical solution, the spatial attention branch captures the statistical features of each spatial location through the concatenation operation of global average pooling and global max pooling, while the frequency attention branch transforms the features to the frequency domain using two-dimensional discrete cosine transform and generates channel-level attention weights in the frequency domain. The cross-enhancement mechanism ensures that when the spatial attention weight map weights the frequency attention features, it confines the spectral channels of the defect features identified in the frequency domain to the region where the defect is located in the spatial domain, suppressing the response of non-defect regions in the spatial domain to the same spectral channel. Conversely, when the frequency attention weight map weights the spatial attention features, it confines the defect locations identified in the spatial domain to the spectral channels corresponding to the defect features in the frequency domain, suppressing the interference of background texture spectral channels at the defect locations. After the results of the two cross-weighting methods are added, the fused feature map retains only the feature components whose spatial location and spectral channel both point to the defect, and the feature components of the background texture that are inconsistent with the defect in at least one dimension in either the spatial or frequency domain are doubly suppressed.

[0023] Optionally, the micro-defect segmentation and post-processing in S4 adopts a two-stage cascaded detection strategy, which includes the following sub-steps: S41. Input the fused feature map into the segmentation decoder and perform the first stage segmentation with a coarse screening threshold. If the coarse screening threshold is lower than the decision threshold of the segmentation decoder, obtain a set of defect candidate regions. S42. For each defect candidate region in the defect candidate region set, perform multiple random inactivation forward propagation during the inference phase to obtain multiple segmentation results, calculate the segmentation probability variance of each pixel in the multiple segmentation results, and use the mean of the segmentation probability variance of each pixel in the defect candidate region as the uncertainty measure of the defect candidate region. S43. In response to the uncertainty metric of the defect candidate region being lower than the uncertainty threshold, the defect candidate region is confirmed as a valid micro-defect; If the uncertainty metric of the defect candidate region is not lower than the uncertainty threshold, the defect candidate region is sent to the fine classification network for the second stage of judgment. The output of the fine classification network determines whether the defect candidate region is a valid micro-defect.

[0024] By employing the above technical solution, the first stage uses a coarse screening threshold lower than the decision threshold for segmentation, resulting in a higher recall rate than when using a standard threshold. This ensures that subtle, low-contrast defects are included in the candidate region set rather than being missed. The second stage involves multiple random inactivation forward propagations to obtain multiple segmentation results for the same region and calculating the pixel-level segmentation probability variance. This variance measures the model's cognitive uncertainty regarding the segmentation result for that region, rather than the classification confidence score, and can distinguish between defect regions that the model is confident in and difficult regions that the model is uncertain about. Regions with low uncertainty are directly identified as valid micro-defects, skipping the second-stage fine-tuning, allowing most clearly identifiable defects to pass quickly with low computational cost. Only difficult regions with high uncertainty are routed to the fine-tuning network for more careful judgment, allocating computational resources as needed to regions that truly require in-depth analysis.

[0025] Optionally, the post-processing of S4 includes morphological constraint processing, in which each constraint parameter is determined by continuous interpolation based on the modal mixing weights: A first constraint parameter set and a second constraint parameter set are preset. The first constraint parameter set corresponds to the constraint parameters when the edge mode weight is at its maximum value. The first constraint parameter set includes aspect ratio constraint parameters and gray-level gradient direction consistency constraint parameters. The second constraint parameter set corresponds to the constraint parameters when the region mode weight is at its maximum value. The second constraint parameter set includes area constraint parameters and gray-level uniformity constraint parameters. For each defect region in the micro-defect segmentation mask, based on the edge mode weight and region mode weight in the modal mixing weight, weighted interpolation is performed on each parameter value in the first constraint parameter set and the corresponding parameter value in the second constraint parameter set to obtain the interpolated constraint parameter set corresponding to the defect region; In response to the defect region's area being lower than the minimum statistically reliable area threshold, the step of determining constraint parameters through continuous interpolation based on the modal mixing weights is skipped. Instead, a fixed set of fallback constraints is applied to the defect region. This set of fallback constraints includes grayscale gradient magnitude threshold constraints and pixel neighborhood contrast threshold constraints. The parameters in the fallback constraint set do not depend on the modal mixing weights.

[0026] By employing the above technical solution, the morphological constraint parameters are continuously interpolated based on the modal mixing weights output by the bimodal attention mechanism in S2. This allows different morphological types of defect regions to obtain constraint parameter combinations that match their morphological characteristics, rather than selecting one from a pre-set set of discrete constraint conditions. Linear defect regions dominated by edge modal weights obtain parameter combinations with strict aspect ratio constraints and high grayscale gradient direction consistency constraints; area-type defect regions dominated by region modal weights obtain parameter combinations with strict area constraints and high grayscale uniformity constraints; and hybrid defect regions obtain interpolation parameter combinations between the two extremes. For defect regions with an area lower than the minimum statistically reliable area threshold, continuous interpolation is skipped, and a fixed fallback constraint set is used. This avoids constraint parameter shifts due to unreliable modal mixing weights when the number of pixels is insufficient to support statistical estimation, ensuring that extremely small defects are not incorrectly removed due to the complexity of post-processing strategies.

[0027] Optionally, in step S43, the input features of the fine classification network are obtained by cropping the local feature map corresponding to the defect candidate region from the fused feature map output by step S3; when cropping the local feature map, the extended context region feature map centered on the defect candidate region is also cropped, and the area of ​​the extended context region feature map is a preset multiple of the area of ​​the defect candidate region. Global average pooling is performed on the extended context region feature map to obtain a context vector. The context vector is then concatenated with the local feature map and input into the fine classification network.

[0028] By adopting the above technical solution, the input features of the fine classification network are obtained by cropping the fused feature map output by S3, reusing the feature information that has already undergone dual-domain attention fusion processing, instead of re-extracting features from the defect candidate region. This reduces the computational cost of the second-stage fine detection from complete feature extraction to the computational cost of cropping and a small number of classification layers. The context vector obtained by global average pooling of the extended context region feature map carries the texture statistics and defect distribution density information around the defect candidate region, enabling the fine classification network to distinguish between isolated real micro-defects and periodically arranged texture pseudo-defects. The latter are difficult to distinguish from real defects in the local feature map, but have statistical characteristics that distinguish them from isolated defects due to their regular arrangement within the context.

[0029] Optionally, before performing S1, a sample set generation step is also included: a. Obtain a sample set of defect-free material surface images and a labeled library of micro-defect morphological statistical features, wherein the micro-defect morphological statistical feature library includes the size distribution, shape distribution, gray-scale contrast distribution, and orientation distribution of various types of micro-defects; b. Based on the distribution parameters in the micro-defect morphological statistical feature library, the morphological parameters of the defect to be synthesized are determined by random sampling; c. Simulate micro-defect regions by randomly selecting locations in an image sample set of defect-free material surfaces and generating them according to determined morphological parameters; d. Perform gradient fusion processing on the boundary of the simulated micro-defect region to obtain synthetic training samples and corresponding segmentation labels.

[0030] By employing the above technical solution, the training data augmentation step randomly samples from an already labeled micro-defect morphological statistical feature library, ensuring that the size, shape, grayscale contrast, and orientation distribution of the synthesized defects are consistent with the statistical distribution of real micro-defects, rather than arbitrarily generating unconstrained pseudo-defects. After generating simulated micro-defect regions at random locations in the defect-free image, a gradient fusion process is performed on the boundaries, resulting in a natural transition between the synthesized defects and the background texture. This avoids boundary abruptness artifacts caused by simple superposition being learned as defect features by the model. This augmentation method expands the amount of training data without requiring additional physical defect sample collection and annotation, alleviating the class imbalance problem caused by the scarcity of micro-defect samples. Attached Figure Description

[0031] Figure 1 This is a flowchart of a deep learning-based method for identifying micro-defects on material surfaces, according to one embodiment of the present invention.

[0032] Figure 2 This is a flowchart of the image enhancement processing steps in one embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of radial power spectral density distribution and main peak detection in one embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram of the directional notch region on the frequency domain feature map in one embodiment of the present invention.

[0035] Figure 5 This is a block diagram of a resolution-preserving multi-scale feature extraction network in one embodiment of the present invention.

[0036] Figure 6 This is a flowchart of the dual-domain attention fusion processing steps in one embodiment of the present invention.

[0037] Figure 7 This is a flowchart of a two-stage cascaded detection strategy in one embodiment of the present invention. Detailed Implementation

[0038] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the application.

[0039] This application provides a deep learning-based method for identifying micro-defects on material surfaces. Addressing the characteristics of micro-defects on material surfaces at the micrometer to sub-millimeter scale—such as low contrast, small area, and easy confusion with background textures—this method employs a collaborative processing approach across four stages: image enhancement, resolution-preserving multi-scale feature extraction, dual-domain attention fusion, and micro-defect segmentation post-processing. The output includes micro-defect identification results such as defect category, defect location coordinates, and defect contour information.

[0040] Micro-defects on material surfaces refer to defects ranging from micrometers to sub-millimeters in size that exist on the surfaces of precision products such as metal plates, semiconductor wafers, and optical components. Common types include scratches, pitting, cracks, and inclusions. Resolution-preserving multi-scale feature extraction networks maintain the same spatial resolution as the input image when extracting multi-scale features across different receptive fields, unlike the progressively decreasing resolution of feature maps in traditional downsampling feature pyramids. The "dual-domain" in dual-domain attention fusion refers to the spatial and frequency domains; the fusion process generates attention weights in both domains and performs cross-enhancement in each.

[0041] To illustrate the specific operations of each step, the following explanation uses the surface inspection of a cold-rolled stainless steel sheet as an example. The surface image of this cold-rolled stainless steel sheet has a resolution of 1024×1024 pixels and is a grayscale image. The steel sheet surface exhibits periodic textures along the rolling direction, with a texture spatial period of approximately 40 pixels. Simultaneously, a second periodic texture, generated by transverse polishing, exists, with a texture spatial period of approximately 15 pixels. Three types of micro-defects are scattered throughout the image: a linear scratch 80 pixels long and 3 pixels wide, with a grayscale difference of approximately 10 gray levels between the scratch area and the surrounding background; a circular pit with a diameter of 12 pixels, with a grayscale difference of approximately 15 gray levels between the pit area and the surrounding background; and an irregular corrosion area with an area of ​​200 square pixels, with a grayscale difference of approximately 8 gray levels between the corrosion area and the surrounding background. Subsequent numerical examples for each step will follow this scenario.

[0042] First, image enhancement processing is performed on the acquired material surface image to improve the visibility of micro-defect signals relative to the background texture. For example... Figure 1 As shown, this method includes four steps, S1-S4.

[0043] S1. Acquire material surface images, perform image enhancement processing on the acquired material surface images, and obtain micro-defect enhanced images.

[0044] Material surface images are acquired on the production line using industrial cameras, and the image format is grayscale. The image enhancement processing in this method is not a general image quality improvement operation, but rather a specific enhancement operation designed to separate micro-defect signals from the background texture. In the original material surface image, the grayscale contrast of micro-defects is typically lower than the grayscale variation of the background texture.

[0045] Taking the aforementioned cold-rolled stainless steel sheet as an example, the grayscale variation caused by the rolling texture in the image is approximately 15 to 20 grayscale levels, while the grayscale difference between the linear scratch and the surrounding background is only about 10 grayscale levels. If the original image is directly fed into the subsequent neural network, the network may misidentify the grayscale variation of the background texture as defect features, or submerge the weak true defect features in the background.

[0046] Therefore, image enhancement processing needs to be performed before feature extraction to improve the grayscale contrast between the micro-defect region and the background region. The input of S1 is the original material surface image, i.e., a grayscale image with a resolution of 1024×1024, and the output is the micro-defect enhanced image. The resolution is the same as the original image, but the grayscale contrast between the defect region and the background region is improved after enhancement, providing a clearer input signal for the subsequent feature extraction of S2.

[0047] Specifically, refer to Figure 2 S1 includes sub-steps S11-S14.

[0048] S11. Obtain a material surface image, perform a two-dimensional fast Fourier transform (FFT) on the material surface image, and obtain a frequency domain feature map.

[0049] The two-dimensional fast Fourier transform converts a 1024×1024 grayscale image in the spatial domain into a frequency domain representation. Each position in the frequency domain feature map corresponds to a combination of spatial frequency and direction, and the amplitude value reflects the energy intensity of that frequency component in the original image.

[0050] S12. Perform radial integration on the amplitude spectrum of the frequency domain feature map to obtain the radial power spectral density distribution, and perform peak detection on the radial power spectral density distribution to obtain at least one main peak frequency.

[0051] like Figure 3 As shown, the radial power spectral density distribution is a one-dimensional curve obtained by integrating the amplitude spectrum of the two-dimensional frequency domain feature map by angle according to the distance from the center of the frequency domain. The horizontal axis is the spatial frequency, and the vertical axis is the cumulative energy at that frequency. Figure 3In the curve, a rapidly decaying background component appears in the low-frequency range, corresponding to the non-periodic grayscale changes on the material surface. Above the background component, the curve exhibits two distinct peaks, marked with dashed lines at their respective frequency positions. The first major peak on the left side of the curve is located at approximately 25.6 periods / 1024 pixels in spatial frequency, with an amplitude of approximately 800, corresponding to a rolling texture with a spatial period of approximately 40 pixels, which has the strongest energy in the material surface image. The second major peak on the right side of the curve is located at approximately 68.3 periods / 1024 pixels in spatial frequency, with an amplitude of approximately 350, corresponding to a polishing texture with a spatial period of approximately 15 pixels, which has weaker energy than the rolling texture. Between the two major peaks and to the right of the second major peak, the curve amplitude is close to zero, indicating that there are no significant periodic texture components in these frequency ranges, and the frequency characteristics of micro-defects are dispersed in these ranges.

[0052] Since there are multiple peak frequencies obtained from peak detection, a cascaded notch filter bank is used to filter the frequency domain feature map. The cascaded notch filter bank includes multiple directional notch filters that correspond one-to-one with the multiple peak frequencies. The notch region of each directional notch filter is a fan-shaped region with the corresponding peak frequency as the radius and the corresponding main direction of the background texture as the center.

[0053] like Figure 4 As shown, in the two-dimensional frequency domain plane, with the frequency domain center O as the origin, the u-axis and v-axis correspond to the horizontal and vertical frequency components, respectively. Dashed circles are drawn with each main peak frequency as the radius, the inner dashed circle corresponding to the first main peak frequency f1, and the outer dashed circle corresponding to the second main peak frequency f2.

[0054] Figure 4 The gray fan-shaped area indicated by label A is the first notch region, located on the inner dashed circle, extending ±15° to both sides of the v-axis (vertical direction) as the center, forming a symmetrical fan shape above and below the frequency domain center O. The first notch region covers the frequency components in the main direction of the rolling texture, and the frequency components in this region are attenuated or set to zero.

[0055] Figure 4 The gray fan-shaped area indicated by the mark C is the second notch region, located on the outer dashed circle. It extends ±15° to both sides of the u-axis (horizontal direction) as the center, forming a symmetrical fan-shaped area to the left and right of the frequency domain center O. The second notch region covers the frequency components in the main direction of the polishing texture.

[0056] Figure 4The area indicated by B is the reserved region, which is the arc portion of the dashed circle not covered by the gray sector. Frequency components within the reserved region are not suppressed. If the frequency characteristic of a micro-defect happens to be located at the same frequency as the main peak but in a different direction, this frequency component falls within the reserved region and will not be falsely suppressed by the directional notch filter. The angular range of the sector region is determined by the energy concentration of the corresponding main peak frequency in the angular domain: if the texture directionality is strong, i.e., the energy is highly concentrated in a certain direction in the angular domain, the sector angular range is narrow; if the texture directionality is weak, the sector angular range is correspondingly wider.

[0057] Continuing with the example of the cold-rolled stainless steel sheet above, the main direction of the rolling texture corresponding to the first main peak is vertical. The first directional notch filter suppresses frequency components within a fan-shaped region centered on the vertical direction at a radius of 25.6 on the frequency domain plane, with a fan angle range of ±15°. The main direction of the polishing texture corresponding to the second main peak is horizontal. The second directional notch filter suppresses frequency components within a fan-shaped region centered on the horizontal direction at a radius of 68.3 on the frequency domain plane.

[0058] Each directional notch filter only suppresses frequency components in the main direction of the corresponding background texture, while retaining frequency components that have the same main peak frequency but different directions. For example, if the frequency characteristic of a micro-defect happens to be at a spatial frequency of 25.6 but in a horizontal direction, the first directional notch filter will not suppress this component because the notch region only covers a vertical range of ±15°, and the horizontal direction is not within this range.

[0059] Multiple directional notch filters are cascaded sequentially on the frequency domain feature map in descending order of their amplitude values ​​in the radial power spectral density distribution corresponding to the main peak frequency. The first directional notch filter is applied first to suppress the rolling texture component with the highest amplitude value, followed by the second directional notch filter to suppress the polishing texture component with the lower amplitude value, thus obtaining the filtered frequency domain feature map. The reason for cascading the filters in descending order of amplitude value is that if the strongest texture component is not eliminated first, its sidelobe expansion may obscure the detection of weaker texture components at adjacent frequencies, leading to a decrease in the suppression accuracy of subsequent filters for weaker textures.

[0060] When peak detection only obtains a single main peak frequency, that is, when there is only a single periodic texture on the material surface, the cascaded notch filter bank degenerates into a single directional notch filter, and directional notching can be performed directly at the main peak frequency.

[0061] S13. Perform a two-dimensional inverse fast Fourier transform on the filtered frequency domain feature map to obtain the frequency domain enhanced image.

[0062] After undergoing a two-dimensional inverse fast Fourier transform, the filtered frequency domain feature map is converted back to the spatial domain, resulting in a frequency domain enhanced image. In the frequency domain enhanced image, the periodic gray-level changes corresponding to the rolling and polishing textures have been suppressed, and the remaining gray-level changes in the image mainly originate from micro-defect signals and a small amount of non-periodic random noise.

[0063] S14. Perform local adaptive contrast enhancement processing on the frequency domain enhanced image to obtain a micro-defect enhanced image.

[0064] Local adaptive contrast enhancement divides the frequency-domain enhanced image into multiple overlapping local windows, for example, each window is 64×64 pixels in size with a 50% overlap between adjacent windows. The mean grayscale value of the pixels within each window is then calculated. and standard deviation When the standard deviation of grayscale within the window Below the preset low contrast threshold At that time, perform contrast stretching on the pixels within the window, and the stretching gain... = / ,in The target standard deviation is [value]. After stretching, the grayscale distribution within the window widens, making localized grayscale anomalies caused by micro-defects more prominent. When the grayscale standard deviation within the window [is...], [the target standard deviation is...]. Not lower than the low contrast threshold At this time, the pixels within the window remain unchanged to avoid over-enhancing areas with sufficient contrast. The processing results of each window are then weighted and averaged in overlapping areas to eliminate discontinuities at window boundaries, ultimately obtaining a micro-defect enhanced image.

[0065] In some embodiments, image preprocessing steps S01-S03 are included before performing S1.

[0066] S01. Obtain a background baseline model constructed from a set of defect-free images of the same material and batch.

[0067] The background baseline model is constructed as follows: Several images, for example 20, are collected from defect-free samples of the same material and batch. The mean value of each pixel in these images is calculated, and the resulting mean map serves as the background baseline model. The background baseline model reflects the grayscale distribution of the background texture on the surface of the material in a defect-free state. Continuing with the example of cold-rolled stainless steel sheets, 20 surface images, manually confirmed to be defect-free, are taken from the same batch of steel sheets. After averaging each pixel, a 1024×1024 mean map is obtained. This mean map preserves the periodic grayscale variation patterns of the rolling and polishing textures.

[0068] S02. Perform a difference operation between the material surface image and the background baseline model to obtain the residual image.

[0069] Differential analysis is a pixel-by-pixel subtraction operation. It subtracts the grayscale value of the corresponding pixel in the background baseline model from the grayscale value of each pixel in the image of the material surface under test; the difference is the residual. In the residual image, the background texture component consistent with the background baseline is eliminated. The remaining signal mainly includes micro-defect signals and residual texture components caused by slight drift in acquisition conditions. Taking the aforementioned cold-rolled stainless steel sheet as an example, after differential analysis, the grayscale variation caused by rolling and polishing textures is reduced from 15 to 20 grayscale levels to approximately 2 to 3 grayscale levels, while the grayscale difference between scratch defects and the background changes from approximately 8 to 12 grayscale levels to approximately 6 to 10 grayscale levels. The signal-to-noise ratio of the micro-defect signal is improved by approximately 5 to 6 times compared to the original image.

[0070] S03. Replace the material surface image with the residual image as the input to S1.

[0071] In the embodiment employing background difference preprocessing, the image enhancement process in S1 uses a multi-channel adaptive filter bank to filter the frequency domain feature map of the residual image, replacing the cascaded notch filter bank described in block 3. The multi-channel adaptive filter bank includes multiple parallel narrowband filters, for example, eight, each covering eight different frequency bands from low to high frequencies. Each narrowband filter independently filters the frequency domain feature map of the residual image, outputting the filtering result for its corresponding frequency band. The outputs of each narrowband filter are weighted and combined using learnable channel weights to obtain the filtered frequency domain feature map. The channel weights are automatically learned during the training phase through end-to-end backpropagation. After training, each channel weight reflects the response intensity pattern of different types of micro-defects in different frequency bands. Taking the aforementioned cold-rolled stainless steel plate as an example, in the trained channel weights, the narrowband filters covering the high-frequency band have higher weights because the edge signals of scratch-like defects are concentrated in the high-frequency band; the narrowband filters covering the mid-frequency band also have higher weights because the spatial frequency characteristics of pitting-like defects are located in the mid-frequency band.

[0072] The difference between the multi-channel adaptive filter bank and the cascaded notch filter bank described in Block 3 is that the cascaded notch filter bank is used for the original image, requiring precise location of each main peak frequency and directional suppression of background textures step by step; the multi-channel adaptive filter bank is used for the residual image after background subtraction, where the energy of the background texture has been significantly reduced. The main task of the filter bank is to selectively enhance the defect signal from the residual signal according to frequency bands, rather than suppressing the background texture. Because the background energy of the residual image is lower, each narrowband filter operates on a signal with a higher signal-to-noise ratio, and the learnable channel weights can more accurately learn the response patterns of different types of micro-defects in different frequency bands. The differential operation and frequency domain filtering are optimized collaboratively during training.

[0073] When it is not possible to obtain defect-free samples from the same batch, such as in the first-piece inspection scenario, the background difference preprocessing steps S01-S03 are not performed. Instead, the cascaded notch filtering method described in Block 3 is directly used to enhance the frequency domain of the original material surface image.

[0074] In some embodiments, the void ratio of each void convolution branch in the parallel void convolution group in S2 is adaptively determined based on the texture spatial period of the material surface image. Specifically, the texture spatial period T of the background texture in the material surface image is obtained based on the radial power spectral density distribution calculated in the frequency domain analysis stage in S1. The void ratio of each void convolution branch is set to a different proportional multiple of the texture spatial period corresponding to the receptive field coverage of each void convolution branch, and the receptive field coverage of each void convolution branch forms a non-integer multiple relationship with the texture spatial period. Taking the above-mentioned cold-rolled stainless steel plate as an example, the spatial period T of the rolling texture is 40 pixels, and the void ratios of the three void convolution branches are set to cover approximately 20 pixels, approximately 60 pixels, and approximately 100 pixels, respectively, corresponding to 0.5T, 1.5T, and 2.5T, all of which are non-integer multiples of T. The purpose of making the receptive field coverage a non-integer multiple of the texture spatial period is to avoid resonance between the receptive field and the texture period. That is, when the receptive field is exactly an integer multiple of the texture period, the convolution operation forms a regular alignment of the sampling positions of the texture peaks and valleys in space, which leads to the superposition and confusion of texture signals and defect signals in the feature response.

[0075] The above describes one implementation of S1, namely, frequency domain enhancement combined with local adaptive contrast enhancement, and a multi-channel adaptive filter bank approach combined with background difference preprocessing. In other embodiments, image enhancement processing can also employ other methods. For example, image enhancement processing can use a spatial domain adaptive histogram equalization method, namely restricted contrast adaptive histogram equalization (CLAHE). CLAHE divides the material surface image into multiple non-overlapping blocks, such as 8×8 blocks, independently calculates the gray-level histogram within each block and performs equalization, and then eliminates discontinuities at block boundaries through bilinear interpolation. Unlike frequency domain filtering, CLAHE directly adjusts contrast at the gray-level distribution level without frequency transformation, and adaptation to different materials does not require adjustment of filtering parameters. Spatial domain adaptive histogram equalization and the aforementioned frequency domain enhancement methods are both implementations of image enhancement processing for material surface images.

[0076] S2. Input the micro-defect enhanced image into the resolution-preserving multi-scale feature extraction network. Extract the multi-scale feature map while keeping the spatial resolution of the input image unchanged. Then, perform attention weighting processing on the multi-scale feature map to obtain the attention-enhanced feature map.

[0077] In traditional multi-scale feature extraction schemes, feature pyramid networks expand the receptive field through progressive downsampling, with feature map resolution decreasing sequentially from 1024×1024 in the input image to 512×512, 256×256, 128×128, or even lower. Micro-defects occupy only a few pixels in the original image; for example, a linear scratch about 3 pixels wide might only have a response area of ​​less than one pixel on a 128×128 resolution feature map. Spatial localization information is gradually attenuated until it is lost during downsampling. Although upsampling and skip connections can partially recover spatial information, the recovered features are mixed with positional offsets and interpolation artifacts introduced in the low-resolution stage, making it impossible to restore the precise boundaries of micro-defects.

[0078] The resolution-preserving multi-scale feature extraction network employs different strategies: all feature maps maintain the same 1024×1024 resolution as the input image, and the receptive field is expanded without reducing resolution through parallel dilated convolutions. The input of S2 is the micro-defect enhancement image output from S1, with a resolution of 1024×1024; the output is the attention enhancement feature map, also with a resolution of 1024×1024, and the number of channels is the sum of the output channels of multiple dilated convolution branches, providing input for the subsequent dual-domain attention fusion in S3.

[0079] like Figure 5 As shown, the resolution-preserving multi-scale feature extraction network includes basic convolutional layers, parallel dilated convolutional groups, and an edge-aware attention module.

[0080] The basic feature map is obtained by performing convolution operations on the micro-defect enhancement image through basic convolutional layers. For example, a standard convolutional layer with a 3×3 convolutional kernel, 64 output channels, a stride of 1, and padding of 1 is used. The resolution of the output basic feature map is 1024×1024×64, which is the same as the spatial resolution of the micro-defect enhancement image.

[0081] The parallel dilated convolution group comprises multiple parallel dilated convolution branches. For example, the parallel dilated convolution group contains three parallel branches, each using a 3×3 convolution kernel with dilation rates of 1, 2, and 4, and each having 64 output channels. The branch with a dilation rate of 1 is equivalent to a standard convolution with a receptive field of 3×3 pixels; the branch with a dilation rate of 2 has a receptive field of 5×5 pixels; and the branch with a dilation rate of 4 has a receptive field of 9×9 pixels. The three branches execute in parallel on the same input, independently extracting features within different receptive fields. The output feature map of each branch has a resolution of 1024×1024×64, the same as the base feature map. The outputs of the three branches are concatenated along the channel dimension to obtain a multi-scale feature map of 1024×1024×192. Through this parallel dilated convolution design, the network can simultaneously perceive the local details of micro-defects (small receptive field branches) and surrounding contextual information (large receptive field branches), without sacrificing spatial resolution for a larger receptive field as in traditional downsampling schemes.

[0082] An edge probability map is obtained by performing learnable edge detection convolution operations on the base feature map through an edge-aware attention module. This learnable edge detection convolution operation differs from traditional fixed-kernel edge detection operators, such as the Sobel or Canny operators. The parameters of its convolution kernel are automatically learned during end-to-end training, making the kernel more responsive to micro-defect edges than to background texture edges. For example, an edge detection convolutional layer using a 3×3 kernel and 1 output channel performs a convolution operation on a 1024×1024×64 base feature map, outputting a 1024×1024×1 edge probability map after passing through a Sigmoid activation function. Each pixel value ranges from 0 to 1, with higher values ​​indicating a greater probability that the pixel is located at a micro-defect edge.

[0083] An edge attention map is obtained by performing diffusion processing on the edge probability map. The diffusion radius of the diffusion processing is a pixel-level adaptive parameter. The diffusion radius of each pixel is determined based on the edge intensity value of the corresponding pixel in the edge probability map. There is an inverse relationship between the diffusion radius and the edge intensity value. The parameter of the inverse relationship is a learnable parameter that is automatically adjusted during training. Continuing with the example of cold-rolled stainless steel sheet, taking a linear scratch as an example, the edge intensity value of the scratch edge pixel is high, for example, 0.9, and the corresponding diffusion radius is small, for example, 2 pixels. After diffusion processing, the attention weight is concentrated on the scratch edge and its neighborhood of 2 pixels above and below it, forming a narrow high-attention area about 7 pixels wide, accurately focusing on the scratch position. Taking a circular pit as an example, the edge intensity value of the pit edge pixel is low, for example, 0.4, and the boundary of the pit is relatively blurred. The corresponding diffusion radius is large, for example, 6 pixels. After diffusion processing, the attention weight covers the pit edge and its surrounding 6-pixel neighborhood, so that the entire pit area and its transition zone are covered by attention. Through this adaptive diffusion mechanism, micro-defects of different scales each obtain an attention coverage range that matches their morphology: micro-defects with clear edges obtain a small diffusion radius to achieve precise focusing, while large-scale defects with blurred edges obtain a large diffusion radius to cover the entire area of ​​the defect.

[0084] The edge attention map is applied to the output feature map of the parallel dilated convolution group. That is, the edge attention map is multiplied element-wise with the output feature map of each dilated convolution branch. This enhances the feature response of the defect edge and its neighboring region, while suppressing the feature response of the large area of ​​normal background region far away from the defect edge, thus obtaining the attention-enhanced feature map.

[0085] In some embodiments, densely connected convolutional layers are appended to the output feature map of dilated convolution branches with high dilation rates to eliminate the checkerboard effect caused by the high dilation rate. The checkerboard effect occurs because dilated convolutions with a dilation rate of 4 sample every 3 pixels, resulting in unsampled pixel locations in the output feature map lacking direct feature information, exhibiting a checkerboard-like discontinuity. The appended densely connected convolutional layers, for example using 1×1 convolutional kernels, learn interpolation from the features of neighboring pixels to complete the feature values ​​at these locations, restoring spatial continuity to the output feature map.

[0086] Furthermore, before determining the diffusion radius driven by the edge probability map, a confidence gating process is included. The purpose of the confidence gating process is to distinguish between true defect edges in the edge probability map and false edges generated by background texture remnants or acquisition noise, preventing false edges from obtaining inappropriate diffusion radii.

[0087] The confidence gating process works as follows: Based on the concentration of gray-level gradient direction distribution among pixels in the neighborhood of each pixel in the basic feature map, an edge confidence index is calculated for each pixel location. Specifically, within a neighborhood window of a given pixel location (e.g., a 5×5 window), the gray-level gradient direction angles of each pixel within the window are calculated. A histogram of these direction angles is then compiled, and the ratio of the most frequent direction interval in the histogram to the total count is calculated. This ratio is used as the edge confidence index.

[0088] At the edges of genuine micro-defects, the gray-level gradient directions of each pixel are highly consistent. Taking the linear scratch mentioned above as an example, the scratch extends horizontally, and the gray-level gradient directions of the pixels at the scratch edge are all perpendicular to the scratch direction, concentrated within a range of approximately 90°±10°. In a 5×5 neighborhood window, the counts in the orientation histogram between 80° and 100° account for approximately 85% of the total counts, with an edge confidence index of 0.85. In contrast, at pseudo-edges caused by residual background texture or acquisition noise, the gray-level gradient directions of each pixel are randomly dispersed and lack consistent directionality. For example, in a 5×5 neighborhood window of a texture residue pseudo-edge, the distribution of the orientation histogram counts in each orientation interval is nearly uniform, with the highest frequency orientation interval accounting for only about 35% of the total counts, resulting in an edge confidence index of 0.35.

[0089] If the edge confidence index of a pixel location is higher than a gating threshold (e.g., a gating threshold of 0.6), the edge probability value of the corresponding pixel location in the edge probability map is passed to the diffusion radius determination stage. In the above-mentioned scratch edge pixel, the edge confidence index is 0.85, which is higher than 0.6. Therefore, the edge probability value of 0.9 for this pixel is normally passed to the diffusion radius calculation, resulting in a small diffusion radius of 2 pixels.

[0090] If the edge confidence index of a pixel location is not higher than the gating threshold, the edge probability value of the corresponding pixel location in the edge probability map is set to zero. The edge confidence index of the aforementioned texture residual pseudo-edge pixel is 0.35, which is lower than 0.6. Therefore, the edge probability value of this pixel is forcibly set to zero, and it no longer participates in the calculation of the diffusion radius, thus not generating a response in the attention map.

[0091] Through confidence-gating, pseudo-edge pixels are excluded from the attention calculation, preventing pseudo-edges from acquiring inappropriately large diffusion radii and incorporating non-defect areas into the attention range. This avoids a vicious cycle where noise signals are amplified, leading to more pseudo-edges, which in turn acquire even larger diffusion radii. The gradient direction consistency metric will be used again in the subsequent morphological constraint processing in S4, but in a different role: here it is used for noise pre-filtering of the attention map during feature extraction, and later for defect region type determination in the post-processing stage, forming a dual verification of front-end pre-filtering and back-end re-verification.

[0092] In some embodiments, the attention-weighted processing in S2 employs a bimodal attention mechanism. This bimodal attention mechanism includes an edge attention modality based on an edge-aware attention module, a region attention modality, and a modality selection classifier.

[0093] The attention map of the edge attention modality is generated through learnable edge detection convolution operations and diffusion processing, consistent with the operation of the edge-aware attention module described above, focusing on linear defect regions with clear boundaries. The attention map of the region attention modality is generated through region response operations with large convolutional kernels. Large convolutional kernels, such as 7×7 kernels, have a large receptive field covering a wide spatial range, producing a strong response to large-area anomalous regions with gradual gray-level changes. The region attention modality is used to weight attention to area-type defect regions with blurred boundaries, such as corrosion and oxidation regions. These defects lack clear boundaries, and the edge detection operations of the edge-aware attention module respond weakly to them, while the region response operations of large convolutional kernels can capture their overall features with gradual gray-level changes.

[0094] The modality selection classifier outputs continuous modality mixing weights based on local features at various spatial locations in the attention-enhanced feature map. For example, a lightweight network consisting of a global average pooling layer, two fully connected layers, and a sigmoid activation function is used. The input is a local feature vector at a specific spatial location in the attention-enhanced feature map, and the output is a two-dimensional vector containing edge modality weights and region modality weights, with the sum of the two weights being 1.

[0095] In its implementation, the bimodal attention mechanism is completed through two rounds of iteration. In the first round, only the single-modal attention of the edge-aware attention module is enabled, and edge attention weighting is applied to the multi-scale feature map to generate a preliminary attention-enhanced feature map. In the second round, the modality selection classifier takes the preliminary attention-enhanced feature map generated in the first round as input, determines the modality mixing weights for each spatial location, and then weights and superimposes the attention maps of the edge attention modality and the region attention modality according to the modality mixing weights to generate a hybrid attention map. This hybrid attention map is then applied to the multi-scale feature map to generate the final attention-enhanced feature map.

[0096] The modal mixing weights are continuous values ​​rather than binary selection, allowing the attention maps of the two modes to be proportionally superimposed and synergistically applied to defect regions of different morphologies. Continuing with the example of the cold-rolled stainless steel sheet, for the linear scratch region, which has clear boundaries and obvious gradient directionality, the edge mode weight output by the modal selection classifier is 0.9, and the region mode weight is 0.1, resulting in a mixed attention map dominated by the edge attention mode. For the irregular corrosion region, with blurred boundaries and gradually changing grayscale, the edge mode weight output by the modal selection classifier is 0.2, and the region mode weight is 0.8, resulting in a mixed attention map dominated by the region attention mode. For a crack penetrating the corrosion region, i.e., a mixed defect region, the edge mode weight output by the modal selection classifier is 0.5, and the region mode weight is 0.5. The attention maps of the two modes are superimposed with equal weights, simultaneously covering the linear edge of the crack and the area characteristics of the corrosion region. The attention map of the edge attention mode and the attention map of the region attention mode are weighted and superimposed according to the modal mixing weights to obtain the mixed attention map.

[0097] Modal mixing weights are passed down to the post-processing stage S4 to drive the determination of constraint parameters in post-processing. These weights not only play a role in the attention map overlay in S2 but also serve as parameter signals passed across steps to S4, enabling the judgment of defect morphology in the feature extraction stage to directly control the selection of constraint strategies in the post-processing stage. The detailed mechanism by which modal mixing weights drive post-processing constraint parameters will be discussed later in the section on morphological constraint processing in S4.

[0098] The above describes one implementation of attention weighting processing in S2, namely a single-modal or dual-modal attention mechanism based on an edge-aware attention module. In other embodiments, attention weighting processing can also employ other methods. For example, a sequential combination of channel attention and spatial attention can be used. Channel attention compresses the spatial dimension through global average pooling and then generates channel-level weights through a fully connected layer, selecting feature channels that strongly respond to micro-defects. Spatial attention compresses the channel dimension through pooling along the channel dimension and then generates spatial-level weights through a convolutional layer, locating the spatial region where the defect is located. Channel attention and spatial attention are executed sequentially, selecting channels first and then locating the space. Unlike the edge-aware attention module, the channel-spatial sequential attention method does not rely on explicit edge detection operations but indirectly focuses on the defect region through channel-level statistics. Both the channel-spatial sequential attention method and the aforementioned edge-aware attention method belong to implementation methods for performing attention weighting processing on multi-scale feature maps.

[0099] S3. Perform dual-domain attention fusion processing on the attention-enhanced feature map to obtain the fused feature map.

[0100] The dual-domain attention fusion process generates attention weights in both the spatial and frequency domains, and uses a cross-enhancement mechanism to constrain and complement the attention information in the two domains, so that the fused feature map can simultaneously capture the positional distribution features of micro-defects in the spatial domain and the spectral features that distinguish it from the background texture in the frequency domain.

[0101] like Figure 6 As shown, S3 includes sub-steps S31-S33.

[0102] S31. Perform global average pooling and global max pooling on the attention enhancement feature map along the channel dimension respectively. After concatenating the pooling results, generate a spatial attention weight map through a convolutional layer. Multiply the spatial attention weight map with the attention enhancement feature map element by element to obtain the spatial attention features.

[0103] Global average pooling compresses the 1024×1024 feature maps of each channel in the attention-enhanced feature map into a single mean value for each channel, while global max pooling compresses the 1024×1024 feature maps of each channel into a single maximum value for each channel. Both methods characterize the feature statistics of each spatial location from the perspectives of average response intensity and peak response intensity, respectively. The results of global average pooling and global max pooling are concatenated along the channel dimension and then passed through a 1×1 convolutional layer and a sigmoid activation function to generate a spatial attention weight map with a resolution of 1024×1024×1, where each pixel value is between 0 and 1. In the spatial attention weight map, the spatial locations of micro-defects have high weights, while large areas of normal background have low weights. Continuing with the example of the cold-rolled stainless steel sheet, the spatial attention weights for scratches and pits are approximately 0.8 to 0.9, for corrosion areas approximately 0.6 to 0.7, and for large areas of normal background approximately 0.1 to 0.2. By multiplying the spatial attention weight map and the attention enhancement feature map element by element, the feature response of the defect region is preserved and enhanced, while the feature response of the background region is attenuated, thus obtaining the spatial attention feature.

[0104] S32. Perform a two-dimensional discrete cosine transform (DCT) on each channel of the attention enhancement feature map, input the transform domain coefficients into the channel attention network to generate a frequency attention weight map, and multiply the frequency attention weight map element-wise with the attention enhancement feature map to obtain the frequency attention features.

[0105] A two-dimensional discrete cosine transform (DCT) converts the spatial domain feature map of each channel to the frequency domain, yielding the DCT coefficient matrix for each channel. In the DCT coefficient matrix, low-frequency coefficients are located in the upper left corner, reflecting the overall grayscale trend of the feature map; high-frequency coefficients are located in the lower right corner, reflecting rapidly changing details in the feature map. A channel attention network performs channel-level weighting on the coefficients at specific frequency positions in the DCT coefficient matrix of each channel. For example, the channel attention network consists of a global average pooling layer and two fully connected layers. The input is the DCT coefficient value of each channel at a specific frequency position, and the output is the frequency attention weight corresponding to each channel. Spectral channels with strong responses to micro-defects receive high weights, while spectral channels with predominantly background texture responses receive low weights. After element-wise multiplication of the frequency attention weight map with the attention-enhanced feature map, the feature responses of the spectral channels corresponding to the defect signal are preserved and enhanced, while the feature responses of the spectral channels dominated by background texture are attenuated, thus obtaining the frequency attention features.

[0106] S33. Weight the frequency attention features with the spatial attention weight map, and weight the spatial attention features with the frequency attention weight map. Add the two weighted results to obtain the fused feature map.

[0107] The cross-enhancement mechanism works as follows: First, the spatial attention weight map is multiplied element-wise with the frequency attention features. This confines the spectral channels of the defect features identified in the frequency domain to the region where the defect is located in the spatial domain. Even if non-defect regions in the spatial domain respond in some spectral channels, the spatial attention weights suppress this. Second, the frequency attention weight map is multiplied element-wise with the spatial attention features. This confines the location of the defect identified in the spatial domain to the spectral channel corresponding to the defect feature. Residual interference from the background texture spectral channels at the defect location is suppressed by the frequency attention weights. After adding the weighted results of the two paths, the fused feature map retains only the feature components whose spatial location and spectral channel both point to the defect. Feature components of the background texture that are inconsistent with the defect in at least one dimension in either the spatial or frequency domain are doubly suppressed.

[0108] Continuing with the example of cold-rolled stainless steel sheet, the residual signal of rolling texture is distributed across a large background area in the spatial domain (low spatial attention weight) and concentrated in a specific spectral channel in the frequency domain (low frequency attention weight), thus being suppressed in both cross-weighted approaches. The scratch defect signal is concentrated at the scratch location in the spatial domain (high spatial attention weight, approximately 0.85) and corresponds to the defect feature spectral channel in the frequency domain (high frequency attention weight), thus being preserved in both cross-weighted approaches. Suppose there is a local variation in the background texture at a certain spatial location with a high spatial attention weight (approximately 0.7), but the spectral characteristics of this variation fall on the spectral channel of the background texture (low frequency attention weight, approximately 0.15). In the second weighted approach, the feature response at this location is suppressed by the frequency attention weight to approximately 0.7 × 0.15, contributing very little to the fused feature map. Through this dual-domain cross-constraint, defect signals and background variations that cannot be distinguished by single-domain attention alone are separated in the fused feature map.

[0109] S4. Perform micro-defect segmentation based on the fused feature map, obtain the micro-defect segmentation mask, perform post-processing steps on the micro-defect segmentation mask, and output the micro-defect recognition result.

[0110] Micro-defect segmentation predicts the probability value of each pixel in the fused feature map belonging to each defect category or the background, outputting a pixel-level segmentation mask. The resolution of the segmentation mask is the same as that of the fused feature map, 1024×1024, and the number of channels is the number of defect categories plus one background class. The micro-defect recognition result includes three components: defect category, i.e., the label of each defect region belonging to categories such as scratches, pitting, corrosion, etc.; defect location coordinates, i.e., the coordinates of the upper left and lower right corners or centroid coordinates of the smallest bounding rectangle of each defect region; and defect contour information, i.e., the pixel-level boundary contour coordinate sequence of each defect region, as well as morphological parameters such as area, aspect ratio, and roundness derived from the contour. The input of S4 is the fused feature map output by S3. S4 contains two sub-steps: micro-defect segmentation and post-processing, which will be elaborated in subsequent paragraphs.

[0111] In some embodiments, the micro-defect segmentation and post-processing in S4 employ a two-stage cascaded detection strategy. For example... Figure 7 As shown, the two-stage cascaded detection strategy includes sub-steps S41-S43.

[0112] S41. Input the fused feature map into the segmentation decoder and perform the first stage segmentation with the coarse screening threshold to obtain a set of defect candidate regions.

[0113] The segmentation decoder, for example, employs a U-Net decoder structure composed of transposed convolutions and skip connections. The input is a 1024×1024 resolution fused feature map output by S3, and the output is a 1024×1024×C probability map, where C is the number of defect categories plus one background category. In the example of the cold-rolled stainless steel sheet, C equals 4, corresponding to the four categories of scratches, pitting, corrosion, and background. In standard use, the segmentation decoder binarizes the probability map using a standard decision threshold. For example, pixels with a defect category probability higher than 0.5 are classified as defects, and pixels with a probability lower than 0.5 are classified as background. The coarse screening threshold is lower than the standard decision threshold of the segmentation decoder; for example, the coarse screening threshold is set to 0.3. When performing the first stage of segmentation using the coarse screening threshold, pixels with a defect category probability between 0.3 and 0.5 are also included in the defect determination range. These pixels correspond to low-confidence defect regions with weak grayscale contrast and weak feature response. Using a coarse screening threshold ensures that the recall rate of the first-stage segmentation is higher than that of the standard decision threshold, so that weak, low-contrast micro-defects are included in the defect candidate region set instead of being missed at this stage.

[0114] Continuing with the example of cold-rolled stainless steel sheet, after performing the first stage of segmentation with a coarse screening threshold of 0.3, five candidate defect regions were obtained. The first candidate region was a linear scratch region with a mean defect category probability of 0.92. The second candidate region was a circular pitting region with a mean probability of 0.78. The third candidate region was an irregular corrosion region with a mean probability of 0.65. The fourth candidate region was a texture residue pseudo-region with a mean probability of 0.42. This region produced a moderate defect category probability response due to incomplete suppression of rolling texture. The fifth candidate region was a noise pseudo-region with a mean probability of 0.35. This region was included by the coarse screening threshold due to grayscale anomalies caused by acquisition noise. If a standard judgment threshold of 0.5 is used, the fourth and fifth candidate regions will be excluded, but some real weak defects with probabilities between 0.3 and 0.5 may also be missed.

[0115] S42. For each defect candidate region in the defect candidate region set, perform multiple random inactivation forward propagation during the inference phase to obtain multiple segmentation results, calculate the segmentation probability variance of each pixel in the multiple segmentation results, and use the mean of the segmentation probability variance of each pixel in the defect candidate region as the uncertainty measure of the defect candidate region.

[0116] Multiple random deactivation forward propagation refers to maintaining the activation state of the Dropout layer in the network during the inference phase and performing T forward propagations on the same input. In each forward propagation, the Dropout layer randomly shuts down different neurons, resulting in slightly different outputs for the same input in T forward propagations. The defect category probabilities of each pixel in the T outputs form a distribution, and the variance of this distribution measures the model's cognitive uncertainty regarding the pixel segmentation result. Cognitive uncertainty differs from classification confidence score: the confidence score is the probability value of the model's output in a single inference, reflecting the model's output conclusion; cognitive uncertainty is the consistency of the model's output under multiple random perturbations, reflecting the model's confidence in its own conclusions.

[0117] The value of T is typically set between 5 and 20. A larger T results in more stable uncertainty estimation, but also increases inference time. For example, T can be set to 10. For each pixel within a defect candidate region, the variance of its defect category probability across 10 segmentation results is calculated. Then, the average variance of all pixels within the defect candidate region is taken as the uncertainty measure for that region.

[0118] Following the examples of the five candidate regions mentioned above, after performing 10 random deactivation forward propagations on each region: the uncertainty metric for the linear scratch region is 0.01, the 10 segmentation results are highly consistent, and the model has high confidence in classifying scratches. The uncertainty metric for the circular pitted region is 0.03. The uncertainty metric for the irregular erosion region is 0.08. The uncertainty metric for the texture residual pseudo-region is 0.25, the 10 segmentation results fluctuate greatly, with some segments classifying it as a defect and others as background, indicating uncertainty in the model's classification of this region. The uncertainty metric for the noisy pseudo-region is 0.31.

[0119] S43. If the uncertainty metric of the defect candidate region is lower than the uncertainty threshold, the defect candidate region is confirmed as a valid micro-defect. If the uncertainty metric of the defect candidate region is not lower than the uncertainty threshold, the defect candidate region is sent to the fine classification network for the second stage of determination.

[0120] An uncertainty threshold of 0.15 was set. The uncertainty metrics for linear scratch areas, circular pitted areas, and irregular corrosion areas were 0.01, 0.03, and 0.08, respectively, all below 0.15, and were directly identified as valid micro-defects, skipping the second-stage fine inspection. These three areas passed with high confidence in the first stage, without consuming computational resources in the second stage. The uncertainty metrics for texture residual pseudo-regions and noise pseudo-regions were 0.25 and 0.31, respectively, both not lower than 0.15, and were sent to the fine classification network for the second-stage judgment. In actual production line inspection, most product surfaces are either defect-free (no candidate areas in stage S41) or have clear defect morphologies (low uncertainty, directly confirmed by S43). The proportion of candidate areas that truly need to enter the second-stage fine inspection is usually no more than 5% to 10% of all candidate areas, allowing computational resources to be allocated as needed to truly challenging areas requiring in-depth analysis.

[0121] The input features of the refined classification network are obtained by cropping the local feature maps corresponding to the defect candidate regions from the fused feature map output by S3. Specifically, the feature tensor corresponding to the minimum bounding rectangle of the defect candidate region is cropped from the fused feature map as a local feature map. The refined classification network directly performs the classification operation on the cropped local feature map, reusing the fused feature information already calculated by S3, instead of re-performing the complete feature extraction process from S1 to S3 for the defect candidate region.

[0122] While cropping the local feature map, an extended context region feature map centered on the defect candidate region is also cropped. The area of ​​the extended context region feature map is a preset multiple of the defect candidate region area, such as 4 times. Specifically, using the minimum bounding rectangle of the defect candidate region as a reference, the rectangle is expanded proportionally in all directions, so that the area of ​​the expanded rectangle is 4 times the area of ​​the original rectangle. The feature tensor corresponding to the expanded rectangular region is cropped from the fused feature map. Global average pooling is performed on the extended context region feature map, compressing the two-dimensional feature tensor into a one-dimensional context vector. The context vector encodes the texture statistics and defect distribution density information around the defect candidate region. The context vector is then concatenated with the local feature map and input into the fine-grained classification network.

[0123] A fine-grained classification network typically consists of 2 to 3 convolutional layers and a fully connected classification head, outputting probability values ​​for each defect category and the background category. When the defect category probability output by the fine-grained classification network is higher than 0.5, the candidate region is confirmed as a valid micro-defect and the corresponding defect category label is output; when the defect category probability is lower than 0.5, the candidate region is discarded.

[0124] Following the two candidate regions routed to the second stage mentioned above, let's illustrate the role of the context vector using a texture residue pseudo-region as an example. The local feature map of the texture residue pseudo-region contains a gray-level anomaly, which, from a local perspective, resembles the features of a real micro-defect. However, in the extended context region clipped by four times the area, multiple morphologically similar regions are observed arranged periodically around this gray-level anomaly. This is a residual manifestation of incompletely eliminated rolling texture. The context vector encodes this periodic arrangement pattern. After receiving the local feature map and the context vector, the fine classification network determines that the texture residue pseudo-region belongs to a texture pseudo-defect rather than a real defect, outputting a defect category probability of 0.12 and discarding the candidate region. Noise pseudo-regions are similarly discarded after the second stage. If a real isolated pit is routed to the second stage due to slightly higher uncertainty, its extended context region does not exhibit a similar periodic arrangement of regions; the context vector reflects random distribution characteristics, which the fine classification network uses to confirm it as a real defect.

[0125] In some embodiments, in continuous inspection scenarios on a production line, a dynamic background model is constructed using a moving average of normal regions from several adjacent frames. The current frame is then subtracted from the dynamic background model and used as input for image enhancement processing. The dynamic background model is continuously updated as the production line runs, enabling it to track the slow drift of acquisition conditions.

[0126] In some embodiments, the post-processing of S4 includes morphological constraint processing, in which each constraint parameter is determined by continuous interpolation based on the modal mixing weights output by the bimodal attention mechanism in S2.

[0127] A first set of constraint parameters and a second set of constraint parameters are preset. Both the first set of constraint parameters and the second set of constraint parameters contain parameters in four dimensions: aspect ratio constraint parameter, gray-level gradient direction consistency constraint parameter, area constraint parameter, and gray-level uniformity constraint parameter. Each parameter takes different values ​​in the two sets of parameters to adapt to the morphological characteristics of linear defects and area-type defects, respectively.

[0128] The first set of constraint parameters corresponds to the constraint parameters when the edge modality weight is at its maximum value, and is suitable for linear defects with clear boundaries. In the first set of constraint parameters, the aspect ratio threshold is set to 10, and the aspect ratio of linear scratches is usually much greater than 10; the gray-level gradient direction consistency threshold is set to 0.7, and the gray-level gradient direction consistency uses the same physical quantity index as the confidence gating processing in block 8, namely the peak concentration of the gray-level gradient direction histogram of each pixel in the defect area, which is used here to determine whether the defect area conforms to the morphological characteristics of a linear defect; the area upper limit threshold is set to 100 square pixels, and the area of ​​linear defects is usually small; the gray-level uniformity standard deviation threshold is set to 40 gray levels, and the gray-level variation inside linear defects is large, so the requirement for gray-level uniformity is lenient.

[0129] The second set of constraint parameters corresponds to the constraint parameters when the modal weight of the region is at its maximum value, and is applicable to area-type defects with blurred boundaries. In the second set of constraint parameters, the aspect ratio threshold is set to 2, as the aspect ratio of area-type defects is usually close to 1; the gray-level gradient direction consistency threshold is set to 0.3, as the gradient direction of area-type defects is dispersed; the area upper limit threshold is set to 500 square pixels, as areas exceeding this area are likely to be large-area texture artifacts rather than local defects; and the gray-level uniformity standard deviation threshold is set to 20 gray levels, as area-type defects such as eroded areas usually exhibit a gradual change in gray-level uniformity.

[0130] For each defect region in the micro-defect segmentation mask, based on the edge mode weights and region mode weights in the modal mixing weights, weighted interpolation is performed on the parameter values ​​in the first constraint parameter set and the corresponding parameter values ​​in the second constraint parameter set to obtain the interpolated constraint parameter set corresponding to the defect region.

[0131] Following the example of the cold-rolled stainless steel sheet above and the modal mixing weight values ​​in block 9, for the linear scratch region, the edge modal weight is 0.9 and the region modal weight is 0.1. Weighted interpolation is performed on the four constraint parameters respectively. The aspect ratio threshold is 0.9×10+0.1×2=9.2. The actual aspect ratio of the linear scratch is 80 / 3≈26.7, which is greater than 9.2, thus satisfying the constraint.

[0132] The grayscale gradient direction consistency threshold is 0.9 × 0.7 + 0.1 × 0.3 = 0.66. The peak concentration of the grayscale gradient direction histogram of each pixel within the linear scratch region is 0.85, which is greater than 0.66, satisfying the constraint. The area upper limit threshold is 0.9 × 100 + 0.1 × 500 = 140 square pixels. The actual area of ​​the linear scratch is 80 × 3 = 240 square pixels, which is greater than 140, thus not satisfying the area upper limit constraint. However, the purpose of the area upper limit constraint is to exclude excessively large texture artifacts. Although the area of ​​the linear scratch exceeds the limit, its aspect ratio and gradient direction consistency highly conform to the characteristics of a linear defect. Therefore, it is still retained as a valid defect in the comprehensive evaluation of the four constraint parameters.

[0133] The standard deviation threshold for grayscale uniformity is 0.9 × 40 + 0.1 × 20 = 38 grayscale levels. The standard deviation of grayscale within the linear scratch region is 32 grayscale levels, which is lower than 38, thus satisfying the constraint. The linear scratch region satisfies three of the four constraint dimensions, and one exceeds the limit but is compensated for by the strong matching of the other three. After comprehensive evaluation, it is retained as a valid defect.

[0134] For irregular eroded regions, the edge mode weight is 0.2, and the region mode weight is 0.8. Weighted interpolation is performed on the four constraint parameters. Aspect ratio threshold: 0.2×10 + 0.8×2 = 3.6. The actual aspect ratio of the eroded region is approximately 1.5, which is lower than 3.6, satisfying the constraint. Gray-scale gradient direction consistency threshold: 0.2×0.7 + 0.8×0.3 = 0.38. The peak concentration of the gradient direction histogram of each pixel within the eroded region is 0.30, which is lower than 0.38, satisfying the constraint (the gradient direction dispersion of area-type defects is a normal morphological feature; consistency below the threshold indicates compliance with the expectation of area-type defects). Area upper limit threshold: 0.2×100 + 0.8×500 = 420 square pixels. The actual area of ​​the eroded region is 200 square pixels, which is lower than 420, satisfying the constraint. The standard deviation threshold for grayscale uniformity is 0.2 × 40 + 0.8 × 20 = 24 grayscale levels. The standard deviation of grayscale within the eroded area is 12 grayscale levels, which is lower than 24, thus satisfying the constraint. Since all four constraint dimensions of the eroded area are satisfied, it is retained as a valid defect.

[0135] In response to a defect region area lower than the minimum statistically reliable area threshold, the step of determining constraint parameters through continuous interpolation based on modal mixing weights is skipped, and a fixed set of fallback constraints is applied to the defect region. The minimum statistically reliable area threshold is set, for example, to 10 square pixels. When the defect region contains fewer than 10 pixels, the number of pixels within the region is insufficient to support statistical estimation of the gray-level gradient direction histogram or gray-level standard deviation, making the continuous interpolation results driven by modal mixing weights unreliable. The fallback constraint set includes a gray-level gradient magnitude threshold constraint and a pixel neighborhood contrast threshold constraint. The gray-level gradient magnitude threshold constraint requires that the average gray-level gradient magnitude of each pixel at the defect region boundary is greater than a preset threshold, for example, greater than 15 gray levels / pixel. The pixel neighborhood contrast threshold constraint requires that the absolute value of the gray-level difference between the center pixel of the defect region and the average of its neighbors is greater than a preset threshold, for example, greater than 10 gray levels. The parameters in the catch-all constraint set are fixed values ​​and do not depend on the modal mixing weights. This ensures that the screening of minute defects is not affected by statistically unreliable modal mixing weights, and avoids the incorrect rejection of minute defects due to the complexity of post-processing strategies.

[0136] In some embodiments, prior to performing S1, a training data augmentation step is also included to increase the number and diversity of micro-defect samples in the training dataset.

[0137] The specific steps for training data augmentation are as follows. First, obtain a sample set of defect-free material surface images and a library of labeled micro-defect morphological statistical features. The images in the sample set of defect-free material surface images only contain background texture and no defects; they can be obtained by filtering from qualified product images collected from the production line. The micro-defect morphological statistical feature library is constructed statistically from existing labeled defect samples, including the size distribution, shape distribution, gray-level contrast distribution, and orientation distribution of various types of micro-defects. Continuing with the example of cold-rolled stainless steel sheet above, assume there are 100 manually labeled scratch defect samples, 80 pitting defect samples, and 50 corrosion defect samples. From the 100 scratch defect samples, the following statistical distributions are obtained: scratch length distribution (mean 60 pixels, standard deviation 20 pixels); width distribution (mean 3 pixels, standard deviation 1 pixel); gray-level contrast distribution (mean 30 gray levels, standard deviation 10 gray levels) between the defect area and the background area; and orientation distribution (scratch direction angle), which is approximately uniformly distributed within the range of 0° to 180°. The statistical characteristics of pitting and corrosion defects were obtained in a similar manner.

[0138] Based on the distribution parameters in the micro-defect morphological statistical feature library, the morphological parameters of the defect to be synthesized are determined by random sampling. For example, when synthesizing a simulated scratch, a length of 75 pixels is randomly sampled from the scratch's length distribution, a width of 2 pixels is randomly sampled from the width distribution, a grayscale contrast of 25 gray levels is randomly sampled from the grayscale contrast distribution, and an orientation angle of 45° is randomly sampled from the orientation distribution. When synthesizing a simulated pit, a diameter of 10 pixels is randomly sampled from the pit's diameter distribution, and a grayscale contrast of 18 gray levels is randomly sampled from the grayscale contrast distribution.

[0139] Simulated micro-defect regions are generated at random locations in a sample set of defect-free material surface images, according to defined morphological parameters. Taking the simulated scratch as an example, a line segment region with a length of 75 pixels, a width of 2 pixels, and an orientation angle of 45° is generated at a random location (e.g., starting at row 300, column 200) in a defect-free 1024×1024 cold-rolled stainless steel sheet surface image. The grayscale value of each pixel within this line segment region is reduced by 25 gray levels to form a simulated scratch.

[0140] A gradient blending process is performed on the boundary of the simulated micro-defect region to obtain synthetic training samples and corresponding segmentation labels. The gradient blending process uses a Gaussian gradient weight to mix the defect grayscale and background grayscale at the boundary of the simulated defect region, with the gradient width set to, for example, 2 pixels. At the boundary pixels, the synthetic grayscale value is a weighted average of the defect grayscale value and the background grayscale value, with the weight decreasing Gaussianly according to the distance from the center of the defect region. The gradient blending process makes the transition between the synthetic defect and the background texture natural, avoiding sharp boundary jumps caused by simple superposition being learned by the model as inherent features of the defect. The corresponding segmentation labels are a label mask with the same resolution as the synthetic training samples; the pixel labels within the simulated defect region are the corresponding defect category labels, and the labels for the remaining pixels are the background category labels.

[0141] Through the above training data augmentation steps, the amount of training data can be expanded without the need for additional physical defect sample collection and annotation. Since the morphological parameters of synthetic defects are sampled from the statistical distribution of real defects, the synthetic samples are consistent with the statistical characteristics of real defects in dimensions such as size, shape, grayscale contrast, and orientation, which can alleviate the class imbalance problem caused by the scarcity of micro-defect samples.

[0142] In some embodiments, an area-aware weighted loss function is used when training the resolution-preserving multi-scale feature extraction network and the segmentation decoder. The area-aware weighted loss function includes a focus loss component and a region overlap loss component. The focus loss component reduces the loss weight for pixels that the model easily classifies correctly and increases the loss weight for pixels that the model struggles to classify correctly. This makes the model's parameter updates during training more driven by hard-to-classify pixels (typically blurry pixels at the edges of micro-defects), enhancing the model's attention to small defect regions. The region overlap loss component directly measures the region overlap between the predicted segmentation mask and the ground truth segmentation label, using the complement of the overlap as the loss value. When the defect region area is much smaller than the background region area, the contribution of background pixels dominates in the pixel-level cross-entropy-based loss function, diluting the contribution of defect pixels. The model tends to predict all pixels as background to obtain a lower total loss. The region overlap loss component is unaffected by the area ratio and directly measures the degree of fit between the predicted mask and the labeled mask on the defect region, mitigating training bias caused by area ratio imbalance.

[0143] In some embodiments, a domain adaptation step is employed when there are differences between the detection environment and the training environment. For example, the model is trained in a laboratory environment under standard lighting conditions. After deployment on a production line, differences in lighting conditions, camera parameters, or material batches can lead to a decrease in the model's detection performance in the target domain. The domain adaptation step involves using the encoder parameters of the resolution-preserving multi-scale feature extraction network trained on source domain data as the teacher network parameters, which remain fixed. Using unlabeled material surface images of the target domain, the target domain images are input into both the teacher and student networks, extracting feature representations at the same level and calculating the feature distribution difference between them. The student network parameters are updated by minimizing this difference, gradually aligning the student network's feature representation in the target domain with the teacher network's feature representation in the source domain. After the update, the encoder part of the resolution-preserving multi-scale feature extraction network is replaced with the student network. This domain adaptation step does not require labeled data for the target domain; adaptation can be achieved using only unlabeled images of the target domain.

[0144] The above describes one implementation of micro-defect segmentation and post-processing in S4, namely a two-stage cascaded detection strategy combined with continuous interpolation morphological constraint processing based on modal mixing weights. In other embodiments, post-processing can also employ other methods. For example, connected component analysis can be directly performed on the micro-defect segmentation mask to obtain each independent defect region. Connected component analysis performs four-connectivity or eight-connectivity labeling on the foreground pixels in the segmentation mask, marking spatially connected foreground pixels as the same independent defect region. Morphological feature parameters such as area, aspect ratio, and average gray-level gradient are calculated for each independent defect region. Defect regions that meet preset fixed constraints are retained as valid micro-defects, while defect regions that do not meet the constraints are discarded. Unlike the two-stage cascaded detection strategy, the direct connected component analysis plus fixed constraint method does not perform a confidence estimation step, does not distinguish between candidate regions that the model is confident in and those that are not, and all candidate regions are selected solely by fixed rules. At the same time, the fixed constraint conditions use the same parameter values ​​for all defect types and do not adaptively adjust according to the defect morphology type. This method has a lower computational cost and is suitable for scenarios with high detection speed requirements but relatively relaxed requirements for the accuracy of difficult sample discrimination. Both the direct connected component analysis with fixed constraints and the two-stage cascaded detection strategy described above are implementation methods for performing post-processing steps on micro-defect segmentation masks.

[0145] Based on the above method embodiments, this application also provides a deep learning-based device for identifying micro-defects on material surfaces. The device includes an image enhancement module, a resolution-preserving multi-scale feature extraction module, a dual-domain attention fusion module, and a micro-defect identification module.

[0146] The image enhancement module is configured to perform image enhancement processing on the acquired material surface image to obtain a micro-defect enhanced image, corresponding to S1 in the above method embodiment. The resolution-preserving multi-scale feature extraction module is configured to extract multi-scale feature maps from the micro-defect enhanced image while maintaining the spatial resolution of the input image, and apply attention weighting processing to the multi-scale feature maps to obtain attention-enhanced feature maps, corresponding to S2 in the above method embodiment. The dual-domain attention fusion module is configured to perform dual-domain attention fusion processing on the attention-enhanced feature maps to obtain fused feature maps, corresponding to S3 in the above method embodiment. The micro-defect recognition module is configured to perform micro-defect segmentation based on the fused feature maps to obtain micro-defect segmentation masks, and perform post-processing on the micro-defect segmentation masks to output micro-defect recognition results. The micro-defect recognition results include defect category, defect location coordinates, and defect contour information, corresponding to S4 in the above method embodiment. The data flow between modules is consistent with the input-output relationship of the above method steps: the output of the image enhancement module serves as the input of the resolution-preserving multi-scale feature extraction module, the output of the resolution-preserving multi-scale feature extraction module serves as the input of the dual-domain attention fusion module, and the output of the dual-domain attention fusion module serves as the input of the micro-defect recognition module.

[0147] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1.A deep learning-based material surface micro-defect identification method, characterized in that, Includes the following steps: S1. Acquire material surface images, perform image enhancement processing on the acquired material surface images, and obtain enhanced images of micro-defects; S2. Input the micro-defect enhanced image into the resolution-preserving multi-scale feature extraction network, extract multi-scale feature maps while maintaining the spatial resolution of the input image, and perform attention weighting processing on the multi-scale feature maps to obtain attention-enhanced feature maps; S3. Perform dual-domain attention fusion processing on the attention-enhanced feature map to obtain the fused feature map; S4. Perform micro-defect segmentation based on the fused feature map, obtain the micro-defect segmentation mask, perform post-processing steps on the micro-defect segmentation mask, and output the micro-defect recognition result, which includes the defect category, defect location coordinates, and defect contour information. 2.The deep learning-based material surface micro-defect identification method of claim 1, wherein, S1 includes the following sub-steps: S11. Obtain a material surface image, perform a two-dimensional fast Fourier transform on the material surface image, and obtain a frequency domain feature map; S12. Radial integration is performed on the amplitude spectrum of the frequency domain feature map to obtain the radial power spectral density distribution. Peak detection is performed on the radial power spectral density distribution to obtain at least one main peak frequency. Each main peak frequency corresponds to the spatial frequency of the periodic background texture in the material surface image. In response to the fact that there are multiple main peak frequencies obtained by peak detection, a cascaded notch filter bank is used to filter the frequency domain feature map. The cascaded notch filter bank includes multiple directional notch filters that correspond one-to-one with the multiple main peak frequencies. The notch region of each directional notch filter is a fan-shaped region with the corresponding main peak frequency as the radius and the corresponding main direction of the background texture as the center. Each directional notch filter only suppresses the frequency component in the corresponding main direction of the background texture and retains the frequency component that is the same as the corresponding main peak frequency but has a different direction. Multiple directional notch filters are cascaded on the frequency domain feature map in descending order of the amplitude value of the corresponding main peak frequency in the radial power spectral density distribution to obtain the filtered frequency domain feature map. S13. Perform a two-dimensional inverse fast Fourier transform on the filtered frequency domain feature map to obtain the frequency domain enhanced image; S14. Perform local adaptive contrast enhancement processing on the frequency domain enhanced image to obtain a micro-defect enhanced image. 3.The deep learning-based material surface micro-defect identification method of claim 1, wherein, Before performing S1, an image preprocessing step is also included: S01. Obtain a background baseline model constructed from a set of defect-free images of the same material and batch; S02. Perform a difference operation between the material surface image and the background baseline model to obtain the residual image; S03. Replace the material surface image with the residual image as the input of S1; In step S1, the image enhancement process uses a multi-channel adaptive filter bank to filter the frequency domain feature map of the residual image. The multi-channel adaptive filter bank includes multiple parallel narrowband filters, each covering a different frequency band. The outputs of each narrowband filter are weighted and combined using learnable channel weights to obtain the filtered frequency domain feature map. 4.The deep learning-based material surface micro-defect identification method of claim 1, wherein, In S2, the resolution-preserving multi-scale feature extraction network includes a basic convolutional layer, a parallel dilated convolutional group, and an edge-aware attention module. The basic convolutional layer performs convolution operations on the micro-defect enhancement image to obtain a basic feature map, the spatial resolution of which is the same as that of the micro-defect enhancement image. The parallel dilated convolutional group includes multiple parallel dilated convolutional branches, and the output feature map of each dilated convolutional branch maintains the same spatial resolution as the basic feature map. The edge-aware attention module is used to perform learnable edge detection convolution operations on the basic feature map to obtain an edge probability map, perform diffusion processing on the edge probability map to obtain an edge attention map, and apply the edge attention map to the output feature map of the parallel dilated convolution group to obtain an attention-enhanced feature map. The diffusion radius of the diffusion process is a pixel-level adaptive parameter. The diffusion radius of each pixel is determined based on the edge intensity value of the corresponding pixel in the edge probability map. The diffusion radius and the edge intensity value are inversely proportional. 5.The deep learning-based material surface micro-defect identification method according to claim 4, characterized in that, Before the determination of the diffusion radius is driven by the edge probability map, the following processing is also included: calculating the edge confidence index of each pixel position based on the concentration of the gray-level gradient direction distribution of each pixel in the neighborhood of each pixel position in the basic feature map; in response to the edge confidence index of the pixel position being higher than the gate threshold, the edge probability value of the corresponding pixel position in the edge probability map is passed to the diffusion radius determination stage; in response to the edge confidence index of the pixel position not being higher than the gate threshold, the edge probability value of the corresponding pixel position in the edge probability map is set to zero. 6.The deep learning-based material surface micro-defect identification method according to claim 5, characterized in that, The attention weighting process in S2 employs a bimodal attention mechanism, which includes an edge attention modality, a region attention modality, and a modality selection classifier based on the edge-aware attention module. The attention map of the edge attention modality is generated through learnable edge detection convolution operations and diffusion processing; The attention map of the region attention mode is generated by the region response operation with a large convolution kernel. The region attention mode is used to perform attention weighting on area-type defect regions with blurred boundaries. The modality selection classifier is used to output continuous modality mixing weights based on the local features of each spatial location in the attention enhancement feature map. The modality mixing weights include edge modality weights and region modality weights. The attention map of the edge attention modality and the attention map of the region attention modality are weighted and superimposed according to the modality mixing weights to obtain a mixed attention map. The modal mixing weights are passed down to the post-processing stage of S4 to drive the determination of constraint parameters in the post-processing. 7.The deep learning-based material surface micro-defect identification method of claim 1, wherein, S3 includes the following sub-steps: S31. Perform global average pooling and global max pooling on the attention enhancement feature map along the channel dimension respectively. After concatenating the pooling results, generate a spatial attention weight map through a convolutional layer. Multiply the spatial attention weight map with the attention enhancement feature map element by element to obtain the spatial attention features. S32. Perform a two-dimensional discrete cosine transform on each channel of the attention enhancement feature map, input the transform domain coefficients into the channel attention network to generate a frequency attention weight map, and multiply the frequency attention weight map with the attention enhancement feature map element by element to obtain the frequency attention features. S33. Weight the frequency attention features with the spatial attention weight map, and weight the spatial attention features with the frequency attention weight map. Add the two weighted results to obtain the fused feature map. 8.The deep learning-based material surface micro-defect identification method of claim 1, wherein, The micro-defect segmentation and post-processing in S4 employs a two-stage cascaded detection strategy, which includes the following sub-steps: S41. Input the fused feature map into the segmentation decoder and perform the first stage segmentation with a coarse screening threshold. If the coarse screening threshold is lower than the decision threshold of the segmentation decoder, obtain a set of defect candidate regions. S42. For each defect candidate region in the defect candidate region set, perform multiple random inactivation forward propagation during the inference phase to obtain multiple segmentation results, calculate the segmentation probability variance of each pixel in the multiple segmentation results, and use the mean of the segmentation probability variance of each pixel in the defect candidate region as the uncertainty measure of the defect candidate region. S43. In response to the uncertainty metric of the defect candidate region being lower than the uncertainty threshold, the defect candidate region is confirmed as a valid micro-defect; If the uncertainty metric of the defect candidate region is not lower than the uncertainty threshold, the defect candidate region is sent to the fine classification network for the second stage of judgment. The output of the fine classification network determines whether the defect candidate region is a valid micro-defect. 9.The deep learning-based material surface micro-defect identification method of claim 6, wherein, The post-processing of S4 includes morphological constraint processing, in which each constraint parameter is determined by continuous interpolation based on the modal mixing weights: A first constraint parameter set and a second constraint parameter set are preset. The first constraint parameter set corresponds to the constraint parameters when the edge mode weight is at its maximum value. The first constraint parameter set includes aspect ratio constraint parameters and gray-level gradient direction consistency constraint parameters. The second constraint parameter set corresponds to the constraint parameters when the region mode weight is at its maximum value. The second constraint parameter set includes area constraint parameters and gray-level uniformity constraint parameters. For each defect region in the micro-defect segmentation mask, based on the edge mode weight and region mode weight in the modal mixing weight, weighted interpolation is performed on each parameter value in the first constraint parameter set and the corresponding parameter value in the second constraint parameter set to obtain the interpolated constraint parameter set corresponding to the defect region; In response to the defect region's area being lower than the minimum statistically reliable area threshold, the step of determining constraint parameters through continuous interpolation based on the modal mixing weights is skipped. Instead, a fixed set of fallback constraints is applied to the defect region. This set of fallback constraints includes grayscale gradient magnitude threshold constraints and pixel neighborhood contrast threshold constraints. The parameters in the fallback constraint set do not depend on the modal mixing weights. 10.The deep learning-based material surface micro-defect identification method of claim 8, wherein, In step S43, the input features of the fine classification network are obtained by cropping the local feature map corresponding to the defect candidate region from the fusion feature map output by step S3; when cropping the local feature map, the extended context region feature map centered on the defect candidate region is also cropped, and the area of ​​the extended context region feature map is a preset multiple of the area of ​​the defect candidate region. Global average pooling is performed on the extended context region feature map to obtain a context vector. The context vector is then concatenated with the local feature map and input into the fine classification network.

Images