A method of fine defect identification and related apparatus

Abstract

The present application provides a kind of fine defect identification method and related equipment, comprising: collecting industrial steel surface original image and corresponding pixel-level defect label mask, obtain training image and label graph;The gradient amplitude and gradient direction of training image are calculated, and are spliced with training image, to obtain enhanced training image;Enhanced training image is input into encoding-decoding segmentation network for processing, to obtain defect foreground probability graph and corresponding prediction mask;According to defect foreground probability graph and label graph, region segmentation loss function, gradient consistency loss function, focal type weighting loss function and boundary perception soft constraint loss function are constructed, to form weighted joint loss function to calculate joint loss value to update the network parameters of encoding-decoding segmentation network in reverse, to obtain fine defect segmentation model;Solve the problem that the prior art exists in complex industrial scene, such as lack of bottom perception, inaccurate middle representation and high-level sample learning imbalance.

Description

Technical Field

[0001] This invention relates to the field of industrial image processing technology, and in particular to a method and related equipment for identifying minute defects. Background Technology

[0002] Surface defects in industrial steel typically exhibit characteristics such as small scale, elongated or fragmented shape, weak contrast, and strong background texture, making them prone to smoothing during downsampling in convolutional networks. Simultaneously, the extreme imbalance between positive and negative samples of defects and background can lead to gradient dominance during training. Against this backdrop, relevant research primarily focuses on multi-scale feature enhancement, label assignment, and fine-grained localization driven by prior cues.

[0003] Multi-scale feature fusion is one of the core methods to improve the ability to identify minute defects. Existing technologies include using multi-task attention networks (M-TA Net) to detect small targets in key components of overhead contact lines; a coarse-to-fine weak object discrimination network (Wod Net) for detecting weak object clouds, which enhances the separability of weak targets through multi-scale feature extraction, cross-layer attention fusion, and cascaded weak object refinement strategies; and the use of a multi-layer interactive CNN-Transformer structure in die-casting defect detection to achieve complementarity between local details and global semantics, thereby improving the model's ability to represent small defects. However, the large number of parameters makes it difficult to meet the real-time requirements of engineering scenarios. To address this, researchers have also lightweighted and structurally modified the YOLOv8 framework for PCB defect detection to balance detection accuracy and inference speed. In addition, for pixel-level minute defects such as scratches on curved metal surfaces, some researchers have achieved finer-grained scratch recognition based on a combination of object detection and semantic segmentation, but its drawback is that it relies too heavily on foreground priors.

[0004] In scenarios involving small targets, traditional anchor box matching and label assignment are prone to problems such as insufficient positive samples and unstable regression. Therefore, anchorless detection and distillation strategies are widely adopted. Existing technologies include a real-time anchorless defect detector that suppresses background noise and enhances small-scale defect features through global and local feature enhancement modules, thereby reducing the false negative rate of small targets. In addition, researchers have proposed a multi-scale anchorless detector that simplifies the model while ensuring detection accuracy through a label localization distillation strategy. Besides the detection framework, to address the gradient imbalance caused by the low proportion of small targets, researchers have proposed variance stabilization loss to improve the stability of segmentation learning for smaller targets. This provides a reference for joint modeling of detection and segmentation of small industrial defects, but the weight setting of the loss term needs to be adjusted according to the specific shape of the defect.

[0005] The methods described above improve the detection performance of subtle defects to some extent through candidate region selection, background noise suppression, and stable training on difficult samples. In the human visual system, there is also a mechanism for selective enhancement and priority processing of salient regions and behavioral information. Related research shows that the receptive field in the visual system is not fixed; attentional modulation can change the processing gain and spatial distribution of the target region in a "spotlight"-like manner. However, most existing industrial defect recognition schemes are still based on fixed convolutional sampling and relatively static local receptive field modeling. For curved, discontinuous, elongated, and irregular defects, they lack morphologically adaptive dynamic perception and region-level focusing, easily leading to strong local responses but insufficient overall structural perception. Furthermore, from a training optimization perspective, the extreme imbalance between a large number of normal background samples and a small number of abnormal defect samples in subtle defect recognition tasks is somewhat similar to the cognitive neuroscience processing idea of ​​"suppressing normal input and highlighting abnormal signals." Sparse coding research shows that visual signals can be characterized by perceptual saliency through more efficient sparse representation methods. In addition, the round-trip interaction between high-level prediction and low-level error signals is an important foundation for the brain to achieve abnormal perception and saliency allocation. In contrast, while existing industrial defect identification methods have partially alleviated the training imbalance problem through strategies such as loss function reweighting and hard sample mining, they lack a unified design that coordinates with low-level structure perception and mid-level region focusing.

[0006] In summary, existing methods for identifying minute defects on the surface of industrial steel generally suffer from the following shortcomings in industrial vision scenarios such as high-speed wire rod and sheet metal: First, minute defects such as cracks, scratches, holes, and inclusions are often characterized by small size, low contrast, blurred boundaries, irregular shapes, and easy obscuring by complex background textures and lighting disturbances, making it difficult for the model to stably lock onto effective defect signals at the low-level perception stage; Second, traditional convolutional networks use fixed-rule sampling and relatively static local receptive fields, lacking sufficient morphological adaptability and region focusing ability for slender, curved, and discontinuous defect types, easily leading to strong local responses but insufficient overall structural modeling; Third, during training, a large number of easily classifiable background samples dominate gradient propagation, causing the model to underlearn key defect samples with blurred boundaries, weak textures, and fragmented shapes, resulting in missed detections, false detections, and inaccurate boundary localization. Summary of the Invention

[0007] This invention provides a method and related equipment for identifying subtle defects, aiming to solve the problems of insufficient low-level perception, inaccurate mid-level representation, and imbalance in high-level sample learning in complex industrial scenarios.

[0008] To achieve the above objectives, the present invention provides a method for identifying minute defects, comprising: Step 1: Collect the original image of the industrial steel surface and the corresponding pixel-level defect annotation mask to obtain the training image and label image; Step 2: Calculate the gradient magnitude and gradient direction of the training image, and then concatenate the training image, gradient magnitude, and gradient direction to obtain the enhanced training image; Step 3: Input the enhanced training image into the encoder-decoder segmentation network for processing to obtain the defect foreground probability map and the corresponding prediction mask; Step 4: Construct a region segmentation loss function, a gradient consistency loss function, a focal weighted loss function, and a boundary-aware soft constraint loss function based on the defect foreground probability map and the label map, forming a weighted joint loss function; Step 5: Calculate the joint loss value using the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain the subtle defect segmentation model; Step 6: Input the surface image of the target industrial steel into the fine defect segmentation model to identify fine defects and obtain the fine defect identification results.

[0009] Furthermore, the expressions for calculating the gradient magnitude and gradient direction of the training image are as follows: ; ; ; in, This represents the gradient magnitude of the training image. This represents the original gradient magnitude. Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction. To prevent extremely small constants with a denominator of zero, The cosine component representing the gradient direction. This represents the sinusoidal component of the gradient direction.

[0010] Furthermore, the encoder-decoder segmentation network includes an encoder for dynamically adjusting the receptive field, a region attention module for spatial focusing enhancement, a decoder for restoring spatial resolution, and a segmentation head for outputting recognition results; The encoder includes a first convolutional layer, a second convolutional layer, a first deformable convolutional unit, and a second deformable convolutional unit connected in sequence, wherein the input of the first convolutional layer is the input of the encoder-decoder segmentation network; The input of the region attention module is connected to the output of the second deformable convolutional unit, and the output of the region attention module is connected to the input of the decoder. The decoder's output is connected to the segmenter's input; The output of the segmentation head is the output of the encoding / decoding segmentation network.

[0011] Furthermore, both the first deformable convolutional unit and the second deformable convolutional unit include an offset prediction convolutional layer, a deformable convolutional layer, a batch normalization layer, and an activation layer connected in sequence.

[0012] Furthermore, the offset prediction convolutional layer is used to generate the offset for each sampling point, and its expression is: ; in, Indicates the offset. This represents the feature map input to the offset prediction convolutional layer. This represents the mapping function corresponding to the offset prediction convolutional layer.

[0013] Furthermore, the region attention module is used for: The input image features are processed through a linear mapping layer to obtain query vector, key vector, and value vector; Attention weights in the height and width directions are obtained based on the query vector, key vector, and value vector. The height axis attention enhancement feature is calculated based on the value vector and the attention weights in the height direction; The width-axis attention enhancement feature is calculated based on the height-axis attention enhancement feature and the width-axis attention weight; After projecting the width axis attention-enhanced features, residual connections and normalization are performed with the input image features to obtain the enhanced features.

[0014] Furthermore, the expression for the weighted joint loss function is: ; in, This represents the weighted joint loss value. This represents the region segmentation loss function. Represents the gradient consistency loss function. This represents the focus-weighted loss function. This represents the boundary-aware soft constraint loss function. , , All of these represent weighting coefficients.

[0015] Furthermore, the expression for the focus-weighted loss function is: ; in, Represents the balance factor. Indicates the focus parameter, This represents the predicted probability that the sample matches the true class.

[0016] The present invention also provides a minor defect identification device, which applies a minor defect identification method. The minor defect identification device includes: The acquisition module is used to acquire raw images of the surface of industrial steel and corresponding pixel-level defect annotation masks to obtain training images and label images. The stitching module is used to calculate the gradient magnitude and gradient direction of the training image, and stitch the training image, gradient magnitude, and gradient direction together to obtain the enhanced training image. The processing module is used to input the enhanced training image into the encoder-decoder segmentation network for processing, and obtain the defect foreground probability map and the corresponding prediction mask; The module is used to construct a region segmentation loss function, a gradient consistency loss function, a focal weighted loss function, and a boundary-aware soft constraint loss function based on the defect foreground probability map and the label map, forming a weighted joint loss function; The update module is used to calculate the joint loss value through the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain the subtle defect segmentation model; The recognition module is used to input the surface image of the target industrial steel into the micro-defect segmentation model to identify micro-defects and obtain the micro-defect recognition results.

[0017] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a method for identifying minute defects.

[0018] The above-described solution of the present invention has the following beneficial effects: This invention acquires raw images of industrial steel surfaces and corresponding pixel-level defect annotation masks to obtain training images and label images. It calculates the gradient magnitude and gradient direction of the training images and concatenates them to obtain enhanced training images. These enhanced training images are then input into an encoder-decoder segmentation network for processing, yielding a defect foreground probability map and a corresponding prediction mask. This allows the network to prioritize locking the edges, textures, and directional structures of defects, forming a low-level sensitivity to subtle anomalies and reducing the adverse effects of complex background textures and lighting changes on defect boundary localization. Finally, a region segmentation loss is constructed based on the defect foreground probability map and the label image. The model employs a weighted joint loss function, consisting of a gradient consistency loss function, a focus-weighted loss function, and a boundary-aware soft constraint loss function. This joint loss value is used to back-update the network parameters of the encoder-decoder segmentation network, resulting in a subtle defect segmentation model. The introduction of focus-weighted loss suppresses the dominant role of training on a large number of redundant background samples, prioritizing the learning of key defect samples. This improves the model's ability to perceive subtle defects under conditions of low contrast, strong background texture, and uneven lighting, thereby addressing the problems of insufficient low-level perception, inaccurate mid-level representation, and imbalanced high-level sample learning in existing technologies in complex industrial scenarios.

[0019] Other beneficial effects of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the encoder-decoder segmentation network in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the minute defect identification device in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the terminal device in an embodiment of the present invention. Detailed Implementation

[0021] To make the technical problems, solutions, and advantages of this invention clearer, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0023] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a locking connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] This invention addresses existing problems by providing a method and related equipment for identifying minute defects.

[0026] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying minute defects, including: Step 1: Collect the original image of the industrial steel surface and the corresponding pixel-level defect annotation mask to obtain the training image and label image; Step 2: Calculate the gradient magnitude and gradient direction of the training image, and then concatenate the training image, gradient magnitude, and gradient direction to obtain the enhanced training image; Step 3: Input the enhanced training image into the encoder-decoder segmentation network for processing to obtain the defect foreground probability map and the corresponding prediction mask; Step 4: Construct a region segmentation loss function, a gradient consistency loss function, a focal weighted loss function, and a boundary-aware soft constraint loss function based on the defect foreground probability map and the label map, forming a weighted joint loss function; Step 5: Calculate the joint loss value using the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain the subtle defect segmentation model; Step 6: Input the surface image of the target industrial steel into the fine defect segmentation model to identify fine defects and obtain the fine defect identification results.

[0027] In this embodiment of the invention, step 1 specifically includes: Acquire raw images of industrial steel surfaces and corresponding pixel-level defect annotation masks; The original images of industrial steel surfaces are preprocessed, including size unification and normalization, to obtain training images. ; Convert the corresponding pixel-level defect annotation mask into a label image. .

[0028] In biological visual systems, retinal ganglion cells possess a lateral inhibition antagonistic mechanism of "excitation center-inhibition periphery." This mechanism does not transmit the input light intensity as is, but rather enhances the system's sensitivity to edges, textures, and contours by amplifying local brightness abrupt changes and suppressing large-area uniform background responses. This mechanism makes the biological visual system insensitive to absolute illumination changes, but highly sensitive to structural boundaries and high-frequency signals. Inspired by this, embodiments of the present invention calculate the gradient magnitude and gradient direction of the training image and stitch the training image, gradient magnitude, and gradient direction together.

[0029] Specifically, the expressions for calculating the gradient magnitude and gradient direction of the training image are as follows: ; ; ; in, This represents the gradient magnitude of the training image. This represents the original gradient magnitude. Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction. To prevent extremely small constants with a denominator of zero, The cosine component representing the gradient direction. This represents the sinusoidal component of the gradient direction.

[0030] In this embodiment of the invention, both the horizontal and vertical gradients are calculated using the Sobel operator, and their calculation expressions are as follows: ; ; in, This represents the grayscale image of the input image. This represents the convolution kernel of the horizontal Sobel operator. This represents the convolution kernel of the vertical Sobel operator. This represents the convolution operation.

[0031] Specifically, the expression for the enhanced training image is obtained by concatenating the training image, gradient magnitude, and gradient direction: ; in, This indicates that the training images have been augmented. Represents training images, This indicates a splicing operation at the channel level.

[0032] In this embodiment of the invention, the intensity information of the original image and the gradient information of the edge are jointly input into the encoder-decoder segmentation network in the above manner, so that the network can obtain explicit structural cues for crack direction, weak boundaries and local abrupt regions before entering deep semantic modeling, thereby forming a low-level structural perception bias for subtle defects.

[0033] The optimal choice is, such as Figure 2 As shown, the encoder-decoder segmentation network includes an encoder for dynamically adjusting the receptive field, a region attention module for spatial focusing enhancement, a decoder for restoring spatial resolution, and a segmentation head for outputting recognition results. The encoder includes a first convolutional layer, a second convolutional layer, a first deformable convolutional unit, and a second deformable convolutional unit connected in sequence, wherein the input of the first convolutional layer is the input of the encoder-decoder segmentation network; The input of the region attention module is connected to the output of the second deformable convolutional unit, and the output of the region attention module is connected to the input of the decoder. The decoder's output is connected to the segmenter's input; The output of the segmentation head is the output of the encoding / decoding segmentation network.

[0034] Since the receptive fields of neurons in the visual system are not fixed but dynamically adjusted according to the target scale, geometry, and task requirements; for small targets, the receptive field tends to shrink to enhance local detail resolution; for long, curved, or irregularly extended targets, the receptive field expands non-uniformly along the main shape of the target. At the same time, under the top-down attention control, the visual system will form a spatial priority allocation for potential target regions like a "spotlight", thereby suppressing irrelevant background and enhancing the representation of salient regions. Inspired by this, this embodiment of the invention introduces deformable convolution in the encoder and a region attention module in the encoder-decoder segmentation network to enhance the region attention of the features processed by the deformable convolution unit.

[0035] In this embodiment of the invention, the calculation formulas for the first convolutional layer and the second convolutional layer are as follows: ; in, This represents the output features of the convolutional layer. This represents the activation function. This indicates a batch normalization operation. This represents a deformable convolution operation. This indicates an enhanced input image.

[0036] Specifically, both the first and second deformable convolutional units include a shift prediction convolutional layer, a deformable convolutional layer, a batch normalization layer, and an activation layer connected in sequence, and their output format is as follows: ; in, This represents the output features of a deformable convolutional unit. This represents the activation function. This indicates a batch normalization operation. This represents a deformable convolution operation based on input features and offsets. This represents the input features of a deformable convolutional unit. This represents the offset of the output of the offset prediction convolutional layer.

[0037] To enable the sampling position to adaptively adjust according to the defect geometry, this embodiment of the invention generates the offset of each sampling point through an offset prediction convolutional layer, the expression of which is: ; in, Indicates the offset. This represents the feature map input to the offset prediction convolutional layer. This represents the mapping function corresponding to the offset prediction convolutional layer.

[0038] In this embodiment of the invention, the expression for the deformable convolutional layer is: ; in, This represents the features output by the deformable convolutional layer. Indicates the output position. Represents the regular convolution sampling grid. Indicates the convolution kernel weights. Indicates the first One sampling point, Indicates the first The offset of each sampling point.

[0039] In this embodiment of the invention, when the sampling position When the coordinates do not fall on integer pixel coordinates, bilinear interpolation is used to obtain the corresponding feature values. In this way, the regular sampling grid can be shifted towards the distribution direction of the target structure, thereby achieving dynamic receptive field adjustment that matches the defect morphology.

[0040] Since fine cracks and scratches often exhibit characteristics of being long, continuous, and extending across regions, it is difficult to establish overall structural connections by relying solely on local convolutions. Therefore, this embodiment of the invention introduces a region attention module in the encoder-decoder segmentation network to achieve spatial priority allocation and long-range dependency modeling.

[0041] Specifically, let the input image features of the region attention module be: ; in, Indicates batch size, Indicates the number of channels. and These represent the height and width of the feature map, respectively.

[0042] First, a feature sequence is constructed along the height direction. For each width position, it is compared with the features of the input image. The corresponding height-direction features are arranged into a one-dimensional sequence, resulting in: ; The input image features are processed through a height-axis linear mapping layer to obtain the query vector, key vector, and value vector in the height direction, expressed as follows: ; in, The query vector representing the height direction. The key vector representing the height direction. A value vector representing the height direction. This represents the input feature sequence in the height direction. , , This represents the learnable parameter matrix in height-axis attention; The attention weights in the height direction are calculated based on the query vector and key vector in the height direction. Its expression is: ; in, Represents the feature dimensions of each attention head; The height-axis attention enhancement feature is calculated based on the attention weights and value vectors in the height direction. Its calculation expression is: ; in, This indicates height axis attention enhancement features; Subsequently, After dimensionality restoration, a height-axis augmented feature map with the same dimensions as the input feature is obtained: ; in, This represents the height axis enhanced feature map. This indicates the height axis dimension recovery operation, used to redirect the height axis attention output from... Restore to ; Feature Map Enhancement Based on Height Axis Construct a feature sequence along the width direction: ; The input image features are processed through a width-axis linear mapping layer to obtain the query vector, key vector, and value vector in the width direction, expressed as follows: ; in, Represents the query vector in the width direction. The key vector represents the width direction. This represents a value vector along the width direction. This represents the feature sequence constructed from the height-axis enhanced feature map along the width direction. , , This represents the learnable parameter matrix in the width-axis attention; The attention weights in the width direction are calculated based on the query vector and key vector in the width direction. Its expression is: ; The width-axis attention enhancement feature is calculated based on the attention weights and value vectors in the width direction. Its calculation expression is: ; in, This indicates attention-enhanced features along the width axis. This represents the attention weights in the width direction. This represents the value vector obtained by mapping the height-axis-enhanced feature map along the width direction; Finally, after projecting the width-axis attention-enhanced features, residual connections and normalization are performed with the input image features of the region attention module to obtain the enhanced features, the calculation expression of which is: ; in, Indicates the enhanced features, This indicates a normalization operation. express Convolution mapping, This indicates a random deactivation operation. This indicates attention-enhanced features along the width axis. This represents the image features input to the region attention module.

[0043] The above process first establishes long-range dependencies along the height axis, and then further establishes spatial relationships along the width axis based on the height axis enhancement features, thereby forming dual-axis regional attention enhancement features, enabling the network to better perceive slender, continuous and cross-regional extended subtle defect structures.

[0044] In the predictive coding theory of cognitive science, the brain continuously makes internal predictions about input signals. For a large number of regular signals that meet expectations, the nervous system reduces their response intensity through inhibition mechanisms to conserve resources. However, for abnormal inputs that deviate from expectations and have large prediction errors, the brain enhances their salience through desuppression and gain amplification, thereby prioritizing subsequent learning and decision-making. In the identification of minute defects on the surface of industrial steel, there is a significant imbalance between a large number of normal background samples and a small number of abnormal defect samples. Moreover, many key defect samples have characteristics such as blurred boundaries, weak textures, and low contrast, making them easily masked by a large number of easily classified background samples during training. To address this, this invention introduces a saliency reweighting mechanism based on prediction error into the loss function, enabling the network to prioritize learning key samples with high prediction errors.

[0045] First, a region segmentation loss function is used to perform basic supervision on the network. For binary classification segmentation tasks, weighted cross-entropy loss can be used as the region segmentation loss function, and its calculation expression is as follows: ; in, This represents the region segmentation loss value. Indicates pixel position, This represents the true label indication value of the i-th sample belonging to class c at pixel position (u,v). This represents the predicted probability that the i-th sample belongs to class c at pixel position (u,v); Then, the gradient consistency loss function is used to constrain the gradient structure of the predicted boundary to be consistent with the true boundary. This allows the network to learn not only the region classification results but also the high-frequency abrupt change patterns of defect boundaries, thereby forming a boundary-sensitive response corresponding to the directional selectivity of biological vision. Its expression is: ; in, The gradient plot, representing the probability map of the defect foreground, is calculated using the following formula: ; The gradient map representing the true label map is calculated as follows: ; in, Represents the gradient magnitude operator; Next, a focus-weighted loss function is used to suppress the gradient contribution of easily classified background samples and amplify the gradient contribution of difficult-to-classify defect samples, so that the model further focuses on key samples with blurred boundaries, weak textures, and large prediction errors. Its expression is: ; ; in, Represents the balance factor. Indicates the focus parameter, This represents the predicted probability that a sample matches the true class. This represents the probability that the model predicts a pixel belongs to a defective foreground. Indicates the true label, Indicates a defective positive sample. Indicates background negative samples; Furthermore, a boundary-aware soft-constraint loss function is introduced to provide more flexible structural supervision to the boundary neighborhood pixels, thereby further enhancing the model's ability to learn contour details. Its expression is: ; ; in, Indicates the first The boundary-aware soft constraint loss value for each sample. This represents the set of pixels in the neighborhood of the label map boundary. Indicates the first Each sample at pixel location The predicted probability of a location belonging to a defective prospect. This indicates the corresponding real label. Indicates boundary soft weights, Indicates pixel position Distance to the real boundary This represents the scale parameter that controls the range of soft weight decay.

[0046] The expression for the weighted joint loss function, constructed by combining the above loss functions, is as follows: ; in, This represents the weighted joint loss value. Indicates batch size, Indicates significance weight, This represents the region segmentation loss function. Represents the gradient consistency loss function. This represents the focus-weighted loss function. This represents the boundary-aware soft constraint loss function. , , All of these represent weighting coefficients.

[0047] In this embodiment of the invention, the significance weight is constructed using the batch average error as a normalization benchmark, and its expression is as follows: ; ; in, Indicates the first Sample-level prediction error for each sample Represents the set of sample pixels. Indicates the first Each sample at pixel location The basic prediction error at that point.

[0048] In this embodiment of the invention, when the prediction error of a certain sample is higher than the average level of the current batch, its corresponding weight... A value greater than 1 indicates that when its prediction error is lower than the batch average, its corresponding weight is... Less than 1.

[0049] To verify the effectiveness of the proposed method in the task of identifying minor defects, the mean intersection-union ratio (NICU), Dice coefficient, precision, and recall were used as evaluation metrics to quantitatively assess the segmentation performance of different methods. The NICU and Dice coefficient measure the overall overlap between the predicted and ground truth regions; higher values ​​indicate more accurate overall segmentation of the defect region. Precision represents the proportion of pixels predicted as defects by the model that are actually defects; higher precision indicates fewer false positives. Recall represents the proportion of real defect pixels successfully identified by the model; higher recall indicates fewer missed detections. The results are shown in Table 1.

[0050] Table 1. Comparison of the effectiveness of various methods in identifying subtle defects.

[0051] As shown in Table 1, the method provided in this embodiment of the invention achieves optimal results in several key indicators. Specifically, the average intersection-over-union (IoU) and Dice coefficient of the method provided in this embodiment are significantly higher than other comparative methods, with the IoU exceeding the baseline model by 0.1109. This indicates that the method of this invention has stronger capabilities in overall defect region localization and segmentation overlap. Regarding precision, the method of this invention achieves an optimal result of 0.7572, significantly higher than other comparative models and exceeding the baseline model by 0.1272, while the recall only decreases by 0.072. This demonstrates that the method provided in this embodiment of the invention has stronger discriminative power in defect prediction and can effectively reduce false detections caused by background textures, noise interference, and spurious defects. This is particularly important for industrial field applications, as excessive false detections directly affect the efficiency of subsequent rejection, alarms, and manual review during actual quality inspection. In comparison, the method provided in this embodiment of the invention achieves better average IoU, Dice coefficient, and precision while maintaining a high recall level, indicating that it achieves a better comprehensive balance between missed detection control and false detection suppression. In summary, the method provided by the embodiments of the present invention is superior to the comparative methods in terms of overall segmentation accuracy, regional consistency and false detection suppression capability, demonstrating stronger performance in identifying subtle defects.

[0052] This invention, in its embodiments, acquires original images of industrial steel surfaces and corresponding pixel-level defect annotation masks to obtain training images and label images. It calculates the gradient magnitude and gradient direction of the training images and concatenates them to obtain enhanced training images. These enhanced training images are then input into an encoder-decoder segmentation network for processing, yielding a defect foreground probability map and a corresponding prediction mask. This allows the network to prioritize locking the edges, textures, and directional structures of defects, forming a low-level sensitivity to subtle anomalies and reducing the adverse effects of complex background textures and lighting changes on defect boundary localization. Based on the defect foreground probability map and label image, a region segmentation loss mechanism is constructed. The loss function, gradient consistency loss function, focal weighted loss function, and boundary-aware soft constraint loss function are combined to form a weighted joint loss function. This joint loss value is used to back-update the network parameters of the encoder-decoder segmentation network, resulting in a subtle defect segmentation model. The focal weighted loss function is introduced to suppress the dominant role of training on a large number of redundant background samples and prioritize the learning of key defect samples. This improves the model's ability to perceive subtle defects under low contrast, strong background texture, and uneven lighting conditions, thereby solving the problems of insufficient low-level perception, inaccurate mid-level representation, and imbalanced high-level sample learning in existing technologies in complex industrial scenarios.

[0053] like Figure 3 As shown, this embodiment of the invention also provides a minor defect identification device 100, which applies a minor defect identification method. The minor defect identification device 100 includes: The acquisition module 101 is used to acquire the original image of the surface of industrial steel and the corresponding pixel-level defect annotation mask to obtain training images and label images; The stitching module 102 is used to calculate the gradient magnitude and gradient direction of the training image, and stitch the training image, gradient magnitude, and gradient direction together to obtain the enhanced training image. Processing module 103 is used to input the enhanced training image into the encoder-decoder segmentation network for processing to obtain the defect foreground probability map and the corresponding prediction mask; Module 104 is used to construct a region segmentation loss function, a gradient consistency loss function, a focal weighted loss function, and a boundary-aware soft constraint loss function based on the defect foreground probability map and the label map, forming a weighted joint loss function; The update module 105 is used to calculate the joint loss value through the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain the subtle defect segmentation model. The recognition module 106 is used to input the surface image of the target industrial steel into the micro-defect segmentation model to identify micro-defects and obtain micro-defect recognition results.

[0054] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0055] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0056] This invention also provides a terminal device, such as... Figure 4 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, it implements the aforementioned method for identifying minor defects.

[0057] The terminal device D10 can be a desktop computer, laptop, handheld computer, server, server cluster, or cloud server, etc. This terminal device may include, but is not limited to, a processor D100 and a memory D101. Those skilled in the art will understand that... Figure 4 This is merely an example of terminal device D10 and does not constitute a limitation on terminal device D10. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0058] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0059] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0060] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0061] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0062] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying minute defects, characterized in that, include: Step 1: Collect the original image of the industrial steel surface and the corresponding pixel-level defect annotation mask to obtain the training image and label image; Step 2: Calculate the gradient magnitude and gradient direction of the training image, and then stitch the training image, the gradient magnitude, and the gradient direction together to obtain the enhanced training image; Step 3: Input the enhanced training image into the encoder-decoder segmentation network for processing to obtain the defect foreground probability map and the corresponding prediction mask; Step 4: Based on the defect foreground probability map and the label map, construct the region segmentation loss function, gradient consistency loss function, focal weighted loss function, and boundary-aware soft constraint loss function to form a weighted joint loss function, the expression of which is: ； in, This represents the weighted joint loss value. Indicates batch size, This represents the significance weight, which is constructed using the batch average error as the normalization benchmark. Indicates the first The region segmentation loss function for each sample Indicates the first The gradient consistency loss function for each sample. Indicates the first A focus-weighted loss function for each sample. , Represents the balance factor. Indicates the focus parameter, This represents the predicted probability that a sample matches the true class. Indicates the first The boundary-aware soft constraint loss value for each sample. , Indicates the first The boundary-aware soft constraint loss value for each sample. This represents the set of pixels in the neighborhood of the label map boundary. Indicates the first Each sample at pixel location The predicted probability of a location belonging to a defective prospect. This indicates the corresponding real label. Indicates boundary soft weights, , , All represent weighting coefficients; Step 5: Calculate the joint loss value using the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain the subtle defect segmentation model. The encoder-decoder segmentation network includes an encoder for dynamically adjusting the receptive field, a region attention module for spatial focusing enhancement, a decoder for restoring spatial resolution, and a segmentation head for outputting the recognition result. The encoder includes a first convolutional layer, a second convolutional layer, a first deformable convolutional unit, and a second deformable convolutional unit connected in sequence, wherein the input end of the first convolutional layer is the input end of the encoder-decoder segmentation network; The input of the region attention module is connected to the output of the second deformable convolutional unit, and the output of the region attention module is connected to the input of the decoder. The output of the decoder is connected to the input of the segmentation head; The output of the segmentation head is the output of the encoder-decoder segmentation network; Step 6: Input the surface image of the target industrial steel into the fine defect segmentation model to identify fine defects and obtain the fine defect identification results.

2. The method for identifying minute defects according to claim 1, characterized in that, The expressions for calculating the gradient magnitude and gradient direction of the training image are as follows: ； ； ； in, This represents the gradient magnitude of the training image. This represents the original gradient magnitude. Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction. To prevent extremely small constants with a denominator of zero, The cosine component representing the gradient direction. This represents the sinusoidal component of the gradient direction.

3. The method for identifying minute defects according to claim 1, characterized in that, Both the first deformable convolutional unit and the second deformable convolutional unit include an offset prediction convolutional layer, a deformable convolutional layer, a batch normalization layer, and an activation layer connected in sequence.

4. The method for identifying minute defects according to claim 3, characterized in that, The offset prediction convolutional layer is used to generate the offset of each sampling point, and its expression is: ； in, Indicates the offset. This represents the feature map input to the offset prediction convolutional layer. This represents the mapping function corresponding to the offset prediction convolutional layer.

5. The method for identifying minute defects according to claim 1, characterized in that, The region attention module is used for: The input image features are processed through a linear mapping layer to obtain query vector, key vector, and value vector; The attention weights in the height direction and the attention weights in the width direction are obtained based on the query vector, the key vector, and the value vector. The height axis attention enhancement feature is calculated based on the value vector and the attention weights in the height direction; The width axis attention enhancement feature is calculated based on the height axis attention enhancement feature and the width direction attention weight; The width axis attention enhancement features are projected and then residually connected and normalized with the input image features to obtain the enhanced features.

6. A device for identifying minute defects, characterized in that, The minor defect identification device, using the minor defect identification method as described in any one of claims 1-5, comprises: The acquisition module is used to acquire raw images of the surface of industrial steel and corresponding pixel-level defect annotation masks to obtain training images and label images. The stitching module is used to calculate the gradient magnitude and gradient direction of the training image, and stitch the training image, the gradient magnitude, and the gradient direction together to obtain an enhanced training image; The processing module is used to input the enhanced training image into the encoder-decoder segmentation network for processing to obtain a defect foreground probability map and a corresponding prediction mask; The construction module is used to construct a region segmentation loss function, a gradient consistency loss function, a focal weighted loss function, and a boundary-aware soft constraint loss function based on the defect foreground probability map and the label map, forming a weighted joint loss function, the expression of which is: ； in, This represents the weighted joint loss value. Indicates batch size, This represents the significance weight, which is constructed using the batch average error as the normalization benchmark. Indicates the first The region segmentation loss function for each sample Indicates the first The gradient consistency loss function for each sample. Indicates the first A focus-weighted loss function for each sample. , Represents the balance factor. Indicates the focus parameter, This represents the predicted probability that a sample matches the true class. Indicates the first The boundary-aware soft constraint loss value for each sample. , This represents the set of pixels in the neighborhood of the label map boundary. Indicates the first Each sample at pixel location The predicted probability of a location belonging to a defective prospect. This indicates the corresponding real label. Indicates boundary soft weights, , , All represent weighting coefficients; The update module is used to calculate the joint loss value through the weighted joint loss function, and update the network parameters of the encoder-decoder segmentation network in reverse based on the joint loss value to obtain a subtle defect segmentation model. The encoder-decoder segmentation network includes an encoder for dynamically adjusting the receptive field, a region attention module for spatial focusing enhancement, a decoder for restoring spatial resolution, and a segmentation head for outputting recognition results. The encoder includes a first convolutional layer, a second convolutional layer, a first deformable convolutional unit, and a second deformable convolutional unit connected in sequence, wherein the input end of the first convolutional layer is the input end of the encoder-decoder segmentation network; The input of the region attention module is connected to the output of the second deformable convolutional unit, and the output of the region attention module is connected to the input of the decoder. The output of the decoder is connected to the input of the segmentation head; The output of the segmentation head is the output of the encoder-decoder segmentation network; The recognition module is used to input the surface image of the target industrial steel into the micro-defect segmentation model to identify micro-defects and obtain micro-defect recognition results.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the minor defect identification method as described in any one of claims 1 to 5.

Images