Pixel-level steel surface defect detection and classification method

Through the improved deep learning model, combining multi-layer feature extraction and side output capture, the model is trained using a hybrid loss function, and various difficulties in steel surface defect detection and segmentation are solved, significantly improving the accuracy and robustness of detection and classification. .

CN119494834AInactive Publication Date: 2025-02-21NANJING UNIV OF INFORMATION SCI & TECH
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202510055720.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-14
Publication Date
2025-02-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art has many difficulties in the detection and segmentation of steel surface defects, including complex defect patterns, noise influence, light and angle changes, imbalance in defect categories, and single selection of loss function, resulting in lower detection and segmentation performance.

Method used

Using an improved deep learning model, including the Backbone layer, the side output layer, the upsampling layer and the Outconv layer, feature extraction is performed through the encoding and decoding paths composed of multiple RSU4F modules and DACONV modules, combined with residual connection and SE attention mechanism. and side output capture. Model training is performed using a mixed loss function of cross entropy loss, Dice loss, and IoU loss.

Benefits of technology

The performance and robustness of steel surface defect detection and segmentation are improved, the model's learning ability to complex defect characteristics and adaptability to different environmental conditions is enhanced, and the accuracy of detection and classification is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119494834A_ABST
    Figure CN119494834A_ABST
Patent Text Reader

Abstract

The invention discloses a pixel-level steel surface defect detection and classification method. The method comprises the following steps: (1) acquiring a plurality of historical steel surface defect images, mask images corresponding to the historical steel surface defect images and actually measured steel surface defect images; (2) constructing an improved # imgabs0 # deep learning model, wherein the improved # imgabs0 # deep learning model comprises a Backbone layer, a side output layer Side, an upper sampling layer and an Outconv layer; (3) training the improved # imgabs1 # deep learning model by adopting a plurality of steel surface defect images and the corresponding mask images to obtain an improved # imgabs2 # steel surface defect detection and classification model; (4) carrying out steel surface defect detection and classification on the plurality of actually measured steel surface defect images by adopting an improved # imgabs3 # steel surface defect detection and classification model; according to the invention, the accuracy of detection and classification is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of steel surface defect detection, and in particular to a pixel-level steel surface defect detection and classification method. Background Art

[0002] The detection and classification of steel surface defects belongs to the task of semantic segmentation. By dividing the background category and the defect category, similar functions to target detection can be achieved, and the accuracy is higher than that of conventional target detection, because semantic segmentation realizes pixel-level classification. Unlike general semantic segmentation tasks, the detection and classification of steel surface defects has more special difficulties. Specifically, the existing technology has the following problems: First, the surface defects of steel are complex and diverse (such as scratches, rust, inclusions, etc.) and have different shapes, which makes it more difficult for the model to identify defects during prediction and it is difficult to capture global information. When multiple types of defects appear on a picture at the same time, the effect of model training is often poor. Second, affected by multiple factors such as shooting media and environment, the obtained steel surface image will have a large number of noise points, which will greatly affect the model's learning of the actual defect features and reduce the robustness of the model. Third, the learning effect of the model on defect features will be affected by image brightness, contrast and the geometric position of the defect. During the shooting process, changes in lighting and materials, as well as changes in shooting angles and orientations will have a significant impact on the recognition and segmentation of steel surface defect images. Due to different shooting environments, images of the same defect will show different grayscale expressions, resulting in poor generalization ability of the model. Fourth, there may be an imbalance in the number of defect categories in a batch of steel surface defect images. Some common defects may occupy a large number of sample pixels, which will cause the model to overfit for these categories of defects, thereby reducing the model's detection and segmentation effects for other rare categories of defects. Fifth, the existing model has a relatively simple choice of loss function during training, making it difficult to adjust the model's parameters. In summary, the existing semantic segmentation model has the defect of low detection and segmentation performance, which in turn affects the problem of inaccurate detection and segmentation of steel surface defects. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a pixel-level steel surface defect detection and classification method to solve the problem of inaccurate existing steel surface defect detection and segmentation.

[0004] Technical solution: The pixel-level steel surface defect detection and classification method described in the present invention comprises the following steps: (1) Obtain multiple historical steel surface defect images and their corresponding steel surface defect mask images as well as measured steel surface defect images; (2) Build an improved The deep learning model includes Backbone layer, side output layer Side, upsampling layer and Outconv layer; (3) Using multiple steel surface defect images and their corresponding mask images to improve Deep learning models are trained to obtain improved Steel surface defect detection and classification model; (4) Adopting improvement-based The steel surface defect detection and classification model performs steel surface defect detection and classification on multiple measured steel surface defect images.

[0005] Furthermore, step (2) is as follows: the Backbone layer is composed of multiple RSU4F modules, each of which includes an encoding and decoding path composed of several DACONV modules; the Side layer performs convolution operations on the output of each RSU4F module to obtain the side output; the upsampling layer uses bilinear interpolation for sampling; the Outconv layer uses a fixed-size convolution kernel to concatenate features. Residual connections are introduced in both the RSU4F module and the DACONV module.

[0006] Furthermore, after data augmentation and denoising of the original steel surface defect images, the Backbone layer is used as the backbone network of the model to extract features from the expanded steel surface defect image set. Without loss of generality, assume that the number of RSU4F operations in the Backbone layer is n , the initial image is n The RSU4F module of the i-th layer extracts features of multiple scales. Specifically, after the image features pass through the RSU4F module of the i-th layer and the global pooling operation, the features are obtained. , and will The above operation is repeated again in the next layer, and finally the deep multi-scale feature information corresponding to the original image is obtained. ; The RSU4F module consists of a convolution layer, an encoding path consisting of 5 layers of DACONV modules with gradually increasing dilated convolution kernels, and a decoding path consisting of 4 layers of DACONV modules with gradually decreasing dilated convolution kernels. Residual connections are introduced in the RSU4F module. Through multi-layer encoding and decoding, deeper abstract features are extracted. The DACONV module refers to a network structure that includes dilated convolution, batch normalization, nonlinear activation, SE attention mechanism, and residual connection.

[0007] Furthermore, the side output layer Side independently processes the features of each stage in the Backbone layer. Perform convolution operation to obtain the corresponding side output , used to capture multi-level global information.

[0008] Furthermore, in the Outconv layer, the feature maps of the steel surface images at different stages are obtained. After splicing, a 1×1 convolution is performed to obtain the desired number of channels.

[0009] Further, improved The deep learning model uses a hybrid loss function that combines cross entropy loss, Dice loss, and IoU loss to calculate the global information loss and update the model parameters; The cross entropy loss function calculation formula is as follows: ; Among them, C is the number of predicted categories; is the one-hot encoding of the true label. For a pixel of category c, if the pixel encounters category c, then , otherwise 0; is the model's predicted probability for category c, calculated by the softmax function; for all pixels in the entire image, the cross entropy loss of each pixel is averaged to obtain the total cross entropy loss; The Dice loss calculation formula is as follows: ; in, Represents the probability value of the i-th pixel predicted by the model; The value of the pixel representing the true label; The IoU loss calculation formula is as follows: ; The IoU loss is based on the intersection-over-union ratio, which measures the ratio of the intersection to the union of the predicted area and the true area, with the goal of maximizing the ratio of the intersection to the union. The calculation formula of the mixed loss function is: ; in, , and are the weights of the three types of loss functions respectively.

[0010] The present invention provides a pixel-level steel surface defect detection and classification system, comprising: Acquisition module: used to acquire multiple historical steel surface defect images and their corresponding steel surface defect mask images and measured steel surface defect images; Building blocks: used to build improved The deep learning model includes Backbone layer, side output layer Side, upsampling layer and Outconv layer; Training module: used to use multiple steel surface defect images and their corresponding mask images to improve Deep learning models are trained to obtain improved Steel surface defect detection and classification model; Classification module: used to adopt improvement-based The steel surface defect detection and classification model performs steel surface defect detection and classification on multiple measured steel surface defect images.

[0011] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the pixel-level steel surface defect detection and classification methods described above is implemented.

[0012] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the pixel-level steel surface defect detection and classification methods described.

[0013] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by splicing multiple RSU4F modules in the Backbone layer, each RSU4F module includes an encoding and decoding path composed of several DACONV modules, which can learn a more complex feature structure; when outputting each RSU4F module in the Side layer, a convolution operation is used to obtain the side output of each RSU4F module, so that the model can better combine low-level and high-level information and alleviate the problem of gradient disappearance; a bilinear interpolation method is used in the upsampling layer to restore the feature maps of different scales to the same resolution as the original input, and the feature maps of different scales are fused; a fixed-size convolution kernel is used in the Outconv layer to splice the features, so that features of multiple scales can be fused, which is convenient for the model to learn features; residual connections are introduced in the RSU4F module and the dilated convolution module to solve the problem of gradient disappearance in the deep network, and at the same time guide the model to pay more attention to the difference of features; a hybrid loss function combining three types of loss functions, namely cross entropy loss, Dice loss and IoU loss, is used in training to form a more robust optimization indicator and optimize the updating method of model parameters. The improved deep learning model improves the detection and segmentation performance. When using the improved steel surface defect detection and classification model to perform steel surface defect detection and classification on multiple measured steel surface defect images, it can better improve the detection, segmentation effect and robustness, and improve the accuracy of detection and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2It is a schematic diagram of the model structure of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0016] like Figure 1 As shown, an embodiment of the present invention provides a pixel-level steel surface defect detection and classification method, comprising the following steps: S1. Acquire multiple historical steel surface defect images and their corresponding mask images, as well as measured steel surface defect images; specifically, obtain multiple historical steel surface defect images using an open source website, or obtain multiple measured steel surface defect images by actually photographing the defective surface of steel on site.

[0017] S2, build an improved deep learning model; e.g. Figure 2 As shown, the improved The deep learning model includes Backbone layer, Side (side output) layer, upsampling layer and Outconv layer; the Backbone layer is composed of multiple RSU4F modules, including encoding and decoding paths composed of several DACONV modules; the Side layer performs convolution operation on the output of each RSU4F module to obtain the side output; the upsampling layer uses bilinear interpolation for sampling; the Outconv layer uses a fixed-size convolution kernel to splice features; residual connections are introduced in both the RSU4F module and the DACONV module.

[0018] The details are as follows: The Backbone layer is the backbone network of the model, which is used to extract features from steel surface defect images. The main body of the RSU4F module is the input convolution layer and the encoding path composed of the DACONV module with a depth of 5 and the decoding path composed of the DACONV module with a depth of 4, and residual connections are introduced in the module. Through multi-layer encoding and decoding, deeper abstract features can be extracted. At the same time, the introduction of residuals alleviates the gradient vanishing problem during model training.

[0019] The main function of the Side (side output) layer is to perform convolution operations on the output of each RSU4F module to obtain the corresponding side output, which can capture richer and more comprehensive multi-level global information and better improve the accuracy and comprehensiveness of model feature extraction.

[0020] The upsampling layer restores the feature map obtained from the side output to the same size as the input through bilinear interpolation, so that it can be fused with other feature maps to obtain a feature image containing global features.

[0021] The Outconv layer uses a convolution kernel of size 1×1 to concatenate features, fuse features of different scales, and integrate semantic information at different levels, so that the output of the model can more accurately segment targets in complex scenes.

[0022] S3, using multiple steel surface defect images and their corresponding mask images to train the improved deep learning model, and obtaining Steel surface defect detection and classification model. It includes the following steps: S31, input the image in the pre-processed steel surface defect image set into the improved The Backbone layer of the deep learning model extracts image features through the RSU4F module. Each stage of extraction can obtain multi-scale feature information of the image. After multiple extractions, the deep multi-scale feature information corresponding to the image can be obtained.

[0023] In some optional implementation schemes, the Backbone layer includes N RSU4F layers, and each RSU4F module includes an encoding and decoding path composed of several DACONV modules; specifically as follows: first, the preprocessed steel surface defect image is input into the first RSU4F module of the Backbone layer of the improved deep learning model for feature extraction, and then a global pooling operation is performed to obtain the original features of the first stage. Among them, the RSU4F module consists of an encoding path composed of a convolution layer, 5 layers of dilated convolution kernels, and a decoding path composed of DACONV modules with gradually increasing dilated convolution kernels, as follows: The steel surface defect image is input to the input convolution layer of the RSU4F module for feature space conversion to obtain the converted feature hx_in. The input convolution layer consists of a DACONV module with an expansion rate of 1. Specifically: Input the image in the expanded steel surface defect image set into the DACONV module and calculate the input residual shortcut. The formula is as follows: ; in, is the convolution weight matrix, is the bias term, and * represents the convolution operation.

[0024] Perform a dilated convolution operation on the input x, and the calculation formula is as follows: ; in, and are the bias term and weight matrix of the convolution operation, respectively. It is the output feature map obtained after this step of operation.

[0025] The input feature map obtained by the previous convolution operation is batch normalized, which includes three steps: calculating the mean and variance, standardization, and linear transformation (scaling and translation). For an input feature map of batch size , assuming the input shape is , for each channel c, first calculate the mean and variance of all samples in the batch, the formula is as follows: ; ; Normalize each sample using the calculated mean and variance: ; in, is the normalized feature map value.

[0026] Finally, we introduce the scaling parameter and translation parameters , used to readjust the standardized features: ; The final result is a batch-normalized feature map. The ReLU activation function is applied to the feature map obtained after batch normalization to introduce nonlinearity to help the model obtain more complex features. The features calculated by the ReLU function are input into the SE attention module to further extract important features and suppress unimportant features.

[0027] The SE attention module includes three steps: Squeeze (global average pooling), Excitation (two-layer fully connected network) and channel weighting. The features calculated by the ReLU function are globally averaged. Specifically, for each channel, its spatial dimension is globally averaged to obtain a scalar representation for each channel. This can be expressed by the following formula: ; in, Represents the first input feature map on channel c pixel values.

[0028] The number of channels is compressed through a fully connected layer to reduce the amount of computation and force the interdependence between learning channels. The formula is: ; in, The shape is The weight matrix of , where c is the compression rate; is the bias vector.

[0029] The ReLU activation function is used to perform nonlinear transformation on the compressed features. The formula is: ; The compressed number of channels is restored to the original number of channels through the second fully connected layer. The formula is: ; Use the Sigmoid activation function to map the weight of each channel to between [0,1], the formula is: ; The weight w is adjusted to a shape that matches the input feature map so that the weight of each channel can be broadcasted in the spatial dimension. The relevant formula is as follows: ; The calculated channel weight w is applied to the feature map x, and the formula is as follows: ; in Represents the output feature map y in channel c and position The value of .

[0030] The output feature map y obtained by the SE attention module is added element by element to the residual shortcut to obtain the converted feature hx_in.

[0031] Next, a DACONV module with an expansion rate of 1 is applied to the input converted feature hx_in to obtain the first layer of encoded features; a DACONV module with an expansion rate of 2 is applied to the first layer of encoded features to obtain the second layer of encoded features; a DACONV module with an expansion rate of 4 is applied to the second layer of encoded features to obtain the third layer of encoded features; a DACONV module with an expansion rate of 8 is applied to the third layer of encoded features to obtain the fourth layer of encoded features; a DACONV module with an expansion rate of 16 is applied to the fourth layer of encoded features to obtain the fifth layer of encoded features.

[0032] After the above 5 layers of encoding, the features are decoded: the fifth layer of encoded features are applied with a DACONV module with an expansion rate of 8 to obtain the fourth layer of decoded features; the fourth layer of encoded features are applied with a DACONV module with an expansion rate of 4 to obtain the third layer of decoded features; the third layer of encoded features are applied with a DACONV module with an expansion rate of 2 to obtain the second layer of decoded features; finally, the second layer of encoded features are applied with a DACONV module with an expansion rate of 1 to obtain the first layer of decoded features.

[0033] The first-layer decoded feature map and the converted feature hx_in are added element by element, and then global pooling is performed to obtain the first-stage feature The formula is as follows: ; in, Represents the i-th sample, c-th channel, position in the output feature map y The value at Represents the i-th sample, c-th channel, position in the input feature map x p and q are the offsets of the pooling window.

[0034] Repeat the above steps to obtain the image features of each stage , and finally the deep multi-scale feature information corresponding to each image in the steel surface defect image set is obtained.

[0035] S32, the deep multi-scale feature information corresponding to each image in the steel surface defect image set (including the original features at each stage) to ) is transmitted to the Side layer, and the side outputs of each stage are generated through convolution operation. to . . S33, the side output of each stage is transmitted to the downsampling layer, and the feature map of the steel surface image at different stages is restored to the same size as the input by the bilinear interpolation method, so as to obtain the feature map of the steel surface image at different stages of the original size. The formula of bilinear interpolation is as follows: ; Among them, b is the batch index, c is the channel index, and i and j represent the spatial coordinates of the output feature map; Represents the interpolation weight, which depends on the distance; Represents the value of the input feature map at the corresponding position.

[0036] S34, the feature map of the steel surface image at different stages of the original size to Transfer to the Outconv layer for splicing and output the desired number of channels through convolution operation. The formula for feature map channel splicing is: ; Among them, B is the batch size, C is the number of channels of each feature map, H and W are the height and width of the feature map respectively. After splicing, the batch size and spatial size of the feature map remain unchanged, and the number of channels becomes times the number of channels of the original single feature map.

[0037] The formula for the convolution operation is: ; in, Represents the value of the output feature map after convolution at the i-th sample, c-th channel, position (j, k); Indicates that the concatenated feature map is in the i-th sample, channels, the value of position (j, k), the channel index From 1 to ; Represents the convolution kernel weight. Here, the convolution kernel size is 1×1 and only linear transformation is performed on the channel. is the bias of the cth output channel.

[0038] S35, compare the output result with the mask corresponding to the steel surface defect image, use the hybrid loss function combining cross entropy loss, Dice loss and IoU loss to calculate the global information loss, and update the model parameters.

[0039] The cross entropy loss function calculation formula is as follows: ; Among them, C is the number of predicted categories; is the one-hot encoding of the true label. For a pixel of category c, if the pixel encounters category c, then , otherwise 0; is the model's predicted probability for category c, calculated by the softmax function; for all pixels in the entire image, the cross entropy loss of each pixel is averaged to obtain the total cross entropy loss; The Dice loss calculation formula is as follows: ; in, Represents the probability value of the i-th pixel predicted by the model; The value of the pixel representing the true label; The IoU loss calculation formula is as follows: ; The IoU loss is based on the intersection-over-union ratio, which measures the ratio of the intersection to the union of the predicted area and the true area, with the goal of maximizing the ratio of the intersection to the union. The calculation formula of the mixed loss function is: ; in, , and are the weights of the three types of loss functions respectively.

[0040] S36. After calculating the global information loss, the Adam optimizer is used to update the model parameters, and the ReduceLROnPlateau scheduler is used to dynamically adjust the model learning rate.

[0041] Step S37: Repeat the above steps S31 to S36 for multiple rounds of training to make the model converge. Steel surface defect detection and classification model.

[0042] Step S4, using the improved The steel surface defect detection and classification model detects and classifies steel surface defects based on multiple measured steel surface defect images. The details are as follows: multiple measured steel surface defect images are input into the improved Deep self-learning is performed in the steel surface defect detection and classification model, and the learning results are output, that is, a mask image of each steel surface defect image is obtained. The pixel value of each pixel in the mask image is the category to which the pixel in the corresponding original image belongs.

[0043] In order to verify the superiority of the method proposed in this invention, a large number of experiments were carried out using the steel defect segmentation dataset NEU-Seg as an effect verification dataset, and the detection effects were compared when different schemes were added (ablation experiments).

[0044] The NEU-Seg dataset contains grayscale images of three typical defects on the steel surface and their corresponding mask images. The task of the experiment is to correctly segment the three types of defects and background pixels to meet the needs of real-time defect detection in actual industry.

[0045] Table 1 shows the experimental results of six network models in the current field on the NEU-Seg dataset. It can be clearly seen from the experimental data that the basic model selected by this invention Not only is the segmentation effect better, but the number of parameters is also very low, with high training speed and operation efficiency, which is very suitable for actual production scenarios. A series of improvements are made based on the network model, making the improved Deep learning models have better results.

[0046] Table 1 Experimental results of six existing semantic segmentation models .

[0047] The embodiment of the present invention also provides a pixel-level steel surface defect detection and classification system, comprising: Acquisition module: used to acquire multiple historical steel surface defect images and their corresponding steel surface defect mask images and measured steel surface defect images; Building blocks: used to build improved The deep learning model includes Backbone layer, side output layer Side, upsampling layer and Outconv layer; Training module: used to use multiple steel surface defect images and their corresponding mask images to improve Deep learning models are trained to obtain improved Steel surface defect detection and classification model; Classification module: used to adopt improvement-based The steel surface defect detection and classification model performs steel surface defect detection and classification on multiple measured steel surface defect images.

[0048] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, any one of the pixel-level steel surface defect detection and classification methods is implemented.

[0049] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the pixel-level steel surface defect detection and classification methods.

Claims

1. A pixel-level steel surface defect detection and classification method, characterized in that: The following steps are involved: (1) Obtain multiple historical steel surface defect images and their corresponding steel surface defect mask images as well as measured steel surface defect images; (2) Build an improved The deep learning model includes Backbone layer, side output layer Side, upsampling layer and Outconv layer; (3) Using multiple steel surface defect images and their corresponding mask images to improve Deep learning models are trained to obtain improved Steel surface defect detection and classification model; (4) Adopting improvement-based The steel surface defect detection and classification model performs steel surface defect detection and classification on multiple measured steel surface defect images.

2. A pixel-level steel surface defect detection and classification method according to claim 1, characterized in that: Step (2) is as follows: the Backbone layer is composed of multiple RSU4F modules, each of which includes an encoding and decoding path composed of several DACONV modules; the Side layer performs convolution operations on the output of each RSU4F module to obtain the side output; the upsampling layer uses bilinear interpolation for sampling; the Outconv layer uses a fixed-size convolution kernel to splice features; residual connections are introduced in both the RSU4F module and the DACONV module.

3. A pixel-level steel surface defect detection and classification method according to claim 2, characterized in that: After data augmentation and denoising preprocessing of the original steel surface defect images, the Backbone layer is used as the backbone network of the model to extract features from the expanded steel surface defect image set; assuming that the number of RSU4F operations in the Backbone layer is n , the initial image is n The RSU4F module of the i-th layer extracts features of multiple scales; specifically, after the image features pass through the RSU4F module of the i-th layer and the global pooling operation, the features are obtained. , and will The above operation is repeated again in the next layer, and finally the deep multi-scale feature information corresponding to the original image is obtained. ; Among them, the RSU4F module consists of an encoding path consisting of a convolution layer, 5 layers of dilated convolution kernels gradually increasing DACONV modules, and a decoding path consisting of 4 layers of dilated convolution kernels gradually decreasing DACONV modules, and residual connections are introduced in the RSU4F module; through multi-layer encoding and decoding, deeper abstract features are extracted; the DACONV module refers to a network structure that includes dilated convolution, batch normalization, nonlinear activation, SE attention mechanism and residual connection performed in sequence.

4. A pixel-level steel surface defect detection and classification method according to claim 2, characterized in that: The side output layer Side independently processes the features of each stage in the Backbone layer Perform convolution operation to obtain the corresponding side output , used to capture multi-level global information.

5. The method for detecting and classifying steel surface defects at pixel level according to claim 2, characterized in that: In the Outconv layer, the feature maps of the steel surface images at different stages are obtained After splicing, a 1×1 convolution is performed to obtain the desired number of channels.

6. A pixel-level steel surface defect detection and classification method according to claim 2, characterized in that: Improved The deep learning model uses a hybrid loss function that combines cross entropy loss, Dice loss, and IoU loss to calculate the global information loss and update the model parameters; The cross entropy loss function calculation formula is as follows: ; Among them, C is the number of predicted categories; is the one-hot encoding of the true label. For pixels of category c, , otherwise 0; is the model's predicted probability for category c, calculated by the softmax function; for all pixels in the entire image, the cross entropy loss of each pixel is averaged to obtain the total cross entropy loss; The Dice loss calculation formula is as follows: ; in, Represents the probability value of the i-th pixel predicted by the model; The value of the pixel representing the true label; The IoU loss calculation formula is as follows: ; The IoU loss is based on the intersection-over-union ratio, which measures the ratio of the intersection to the union of the predicted area and the true area, with the goal of maximizing the ratio of the intersection to the union. The calculation formula of the mixed loss function is: ; in, , and are the weights of the three types of loss functions respectively.

7. A pixel-level steel surface defect detection and classification system, characterized in that: include: Acquisition module: used to acquire multiple historical steel surface defect images and their corresponding steel surface defect mask images and measured steel surface defect images; Building blocks: used to build improved The deep learning model includes Backbone layer, side output layer Side, upsampling layer and Outconv layer; Training module: used to use multiple steel surface defect images and their corresponding mask images to improve Deep learning models are trained to obtain improved Steel surface defect detection and classification model; Classification module: used to adopt improvement-based The steel surface defect detection and classification model performs steel surface defect detection and classification on multiple measured steel surface defect images.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, a pixel-level steel surface defect detection and classification method according to any one of claims 1 to 6 is implemented.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a pixel-level steel surface defect detection and classification method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Defect detection model training method, defect detection method and related device

    CN111814867A

  • Concrete apparent crack segmentation method based on deep learning and image processing

    CN113744185A

  • Steel surface defect detection method based on deep learning

    CN116363124A

  • Ceramic tile surface defect segmentation method based on improved U2-Net

    CN116645514A

  • Tumor segmentation method and system for breast ultrasound image, medium and terminal

    CN116703945A