Industrial product appearance defect semantic segmentation method and system

Abstract

The invention discloses an industrial product appearance defect semantic segmentation method and system, and relates to the technical field of product defect detection, and the method comprises the steps: making a data set of an appearance image of a to-be-detected industrial product; performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set; constructing a semantic segmentation model, and performing multiple iterative training on the semantic segmentation model through the training set by using a training mode based on edge supervision and adopting a combined weighted network loss function until the network loss function reaches convergence to obtain a trained semantic segmentation model; inputting the test set into a trained semantic segmentation model for testing to obtain a final semantic segmentation model; and inputting a to-be-detected image into the final semantic segmentation model, and outputting an industrial product appearance defect semantic segmentation result. According to the invention, the precision and accuracy of image segmentation are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of product defect detection, and more specifically, to a method and system for semantic segmentation of appearance defects of industrial products. Background Art

[0002] Surface defect detection is widely used in various industrial fields, including 3C, semiconductors and electronics, automobiles, chemicals, medicine, light industry, military industry and other industries. According to statistics, more than 80% of industrial surface defect detection currently still relies on manual detection methods. Manufacturing companies represented by Foxconn and Bern Optics recruit a large number of quality inspection workers and conduct inspections in the form of assembly lines. However, manual inspection methods have the disadvantages of high labor intensity, poor detection stability and consistency, low degree of automation, low production efficiency, and difficulty in forming lean production, which makes it difficult to cover large-scale quality inspection needs. In recent years, with the emergence of new technologies in the fields of industrial imaging, computer vision and deep learning, vision-based industrial defect detection technology has made great progress and has become an effective solution for product appearance quality inspection, which has aroused strong attention from academia and industry.

[0003] Semantic segmentation is a technology in the field of computer vision. Its goal is to correctly classify each pixel in a digital image or video frame into a predefined category. Through semantic segmentation, different objects and regions in the image can be distinguished, and each object can be identified and located. Among them, U-Net is a deep learning model designed specifically for semantic segmentation tasks. U-Net adopts a commonly used convolutional neural network architecture, which consists of an encoder and a decoder. Its architecture is simple and scalable, and it is widely used in various application scenarios. However, its full convolutional network structure has certain limitations. The traditional model cannot efficiently extract contextual information between multiple levels without ensuring accurate positioning, resulting in it being insensitive to local information and easily ignoring boundary detail information, which affects the segmentation accuracy. At present, the semantic segmentation model based on U-Net has achieved remarkable results in industrial product appearance defect detection, but there are still some problems and shortcomings, such as sensitivity to noise interference, difficulty in distinguishing defects from background, and difficulty in handling complex boundaries. Summary of the invention

[0004] In order to solve the problems of low precision and accuracy of image segmentation in the prior art, a method and system for semantic segmentation of appearance defects of industrial products are proposed to effectively improve the precision and accuracy of image segmentation.

[0005] The technical solution of the present invention is as follows:

[0006] The present invention discloses a method for semantic segmentation of appearance defects of industrial products, comprising the following steps:

[0007] S1. Prepare a dataset of appearance images of industrial products to be tested;

[0008] S2, performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set;

[0009] S3, constructing a semantic segmentation model, using an edge-supervised training method, adopting a jointly weighted network loss function, and iteratively training the semantic segmentation model through the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model;

[0010] S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation result of the appearance defects of industrial products.

[0011] Preferably, the data set for preparing the appearance image of the industrial product to be inspected includes:

[0012] S11, obtaining an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, comparing the abnormal sample with the normal sample image, and determining the defect type of the abnormal sample;

[0013] S12. Use an image annotation tool to perform defect annotation on abnormal samples of determined defect types to obtain abnormal defect images, and assign labels of corresponding defect types to different defect targets in the abnormal defect images;

[0014] S13. The abnormal defect images and the labels are made into a data set.

[0015] Preferably, the data preprocessing of the data set includes:

[0016] S21. Performing data enhancement on the data set to obtain a data enhanced data set;

[0017] S22. Divide the data-enhanced dataset into a training set and a test set according to a set ratio.

[0018] Preferably, the semantic segmentation model comprises a ResNet-50 encoder, an ECA attention module and an FPN feature pyramid decoder connected in sequence.

[0019] Preferably, the ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input into the semantic segmentation model; it includes a total of five convolutional layers, each of which is connected to a layer of rectified linear unit and a maximum pooling operation.

[0020] Preferably, the ECA attention module includes a global average pooling layer, a 1D convolution layer and a Sigmoid activation function layer;

[0021] The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses the 1D convolution operation with efficient cross-channel information acquisition capability to extract feature pixels of the one-dimensional feature vector. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result. The weighted result is connected to the one-dimensional feature vector through a jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder.

[0022] Preferably, the FPN feature pyramid decoder includes several layers of decoders, and the FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top, and the output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer.

[0023] Preferably, the edge-supervised training method includes: using a Laplacian operator edge extraction algorithm to extract image edge information of the label image, obtaining image edge feature maps of different scales through convolutions of different step lengths, upsampling them to their original size, and fusing them to achieve dynamic reshaping.

[0024] Preferably, the network loss function using joint weighting includes:

[0025] Calculate the edge detail loss function under the guidance of edge information supervision;

[0026] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0027] L detail =L dice +L nce

[0028] Among them, L detail represents the edge detail loss function calculated for the label feature map, L dice represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0029] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0030] L main =L dice +L ce +L focal

[0031] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, L focal represents the focal loss function;

[0032] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0033] loss = λL detail +μL main

[0034] Among them, loss represents the network loss function, λ represents the first weighted parameter, and μ represents the second weighted parameter.

[0035] The present invention also proposes a semantic segmentation system for industrial product appearance defects, comprising:

[0036] A production module, used for producing a data set of appearance images of industrial products to be tested;

[0037] A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set;

[0038] A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function, and iteratively trains the semantic segmentation model through the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; and inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model;

[0039] The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

[0040] Beneficial effects of the present invention:

[0041] The present invention proposes a method and system for semantic segmentation of industrial product appearance defects. First, a semantic segmentation model of industrial product appearance defects is constructed. Then, a training method based on edge supervision is used to incorporate edge supervision into the training. A jointly weighted network loss function is adopted to perform multiple iterative training on the semantic segmentation model to help the semantic segmentation model better learn the boundary information of the object, and improve the perception and learning ability of the semantic segmentation model for defect details in the shallow layer, so as to obtain a trained semantic segmentation model. The test set is further input into the trained semantic segmentation model for testing to obtain a final semantic segmentation model. The final semantic segmentation model is used to output the semantic segmentation result of the industrial product appearance defects, so as to effectively improve the precision and accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a method for semantic segmentation of appearance defects of industrial products provided in an embodiment.

[0043] Figure 2 A schematic diagram of the specific process of the method for semantic segmentation of industrial product appearance defects provided in an embodiment.

[0044] Figure 3 A schematic diagram of a semantic segmentation model provided for an embodiment.

[0045] Figure 4 A schematic diagram of an edge-supervised training method provided in an embodiment.

[0046] Figure 5 An example diagram of a semantic segmentation result provided by an embodiment.

[0047] Figure 6 A schematic diagram of the structure of a semantic segmentation system for industrial product appearance defects provided in an embodiment. DETAILED DESCRIPTION

[0048] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] In this embodiment, see Figure 1 and Figure 2 , a semantic segmentation method for appearance defects of industrial products, comprising the following steps:

[0051] S1. Prepare a dataset of appearance images of industrial products to be tested;

[0052] S2, performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set;

[0053] S3. Construct a semantic segmentation model, use an edge-supervised training method, adopt a jointly weighted network loss function, and iterate the semantic segmentation model multiple times through the training set until the network loss function converges to obtain a trained semantic segmentation model; input the test set into the trained semantic segmentation model for testing to obtain a final semantic segmentation model; wherein the semantic segmentation model includes a ResNet-50 encoder, an ECA attention module, and an FPN feature pyramid decoder connected in sequence.

[0054] S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation result of the appearance defects of industrial products.

[0055] In this embodiment, the ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input into the semantic segmentation model; it includes a total of five convolutional layers, each of which is connected to a layer of rectified linear unit and a maximum pooling operation.

[0056] More specifically, the ECA attention module can efficiently obtain the correlation information between channels, which is used to guide the shallow features of the network to propagate to the deep layers, and improve the model's ability to process local and global information. The ECA attention module includes a global average pooling layer, a 1D convolution layer, and a Sigmoid activation function layer;

[0057] The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses the 1D convolution operation with efficient cross-channel information acquisition capability to extract the feature pixels of the one-dimensional feature vector. The convolution kernel size of the 1D convolution affects the number of channels for calculating each weight in the attention mechanism. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result, so that the network can focus more on the channels that need attention. The weighted result is connected to the one-dimensional feature vector through jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is transmitted to the FPN feature pyramid decoder.

[0058] More specifically, the convolution kernel size k of 1D convolution is selected as follows:

[0059] Convolution kernel size k: C represents the number of channels of input data;

[0060] More specifically, the FPN feature pyramid decoder includes several layers of decoders. The FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer. By fusing features of different scales, multi-scale segmentation prediction is achieved.

[0061] More specifically, the image features extracted by the ResNet-50 encoder are aggregated and restored to their original size through upsampling of the FPN feature pyramid decoder and skip cascading of the ECA attention module to facilitate visualization of the segmentation prediction results. By aggregating semantic features of different scales, the model can effectively capture a wider range of global information and improve the global receptive field.

[0062] In this embodiment, the edge-supervised training method includes: using OpenCV's Laplacian operator edge extraction algorithm to extract image edge information of the label image, obtaining image edge feature maps of different scales through convolutions of different step lengths, upsampling them to their original size, and fusing them to achieve dynamic reshaping.

[0063] More specifically, the network loss function using joint weighting includes:

[0064] Calculate the edge detail loss function under the guidance of edge information supervision;

[0065] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0066] L detail =L dice +L bce

[0067] Among them, L detail represents the edge detail loss function calculated for the label feature map, L dice represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0068] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0069] L main =L dice +L ce +L focal

[0070] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, Lfocal represents the focal loss function;

[0071] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0072] loss = λL detail +μL main

[0073] Among them, loss represents the network loss function, λ represents the first weighted parameter, and μ represents the second weighted parameter.

[0074] In this embodiment, the test set is input into the trained semantic segmentation model for testing. If the defect type of the output semantic segmentation result is correct and different defect types are marked with different colors in the figure, the test is successful. Otherwise, the test fails and iterative training continues.

[0075] In this embodiment, firstly, the ResNet-50 encoder is used to optimize the problems of gradient vanishing and model degradation of the neural network, accelerate the network training speed, and improve the accuracy of image segmentation and the generalization ability of the semantic segmentation model; secondly, by introducing the channel attention mechanism of the ECA attention module, the network is more focused on the channels that need attention; then, through the FPN feature pyramid decoder, the inter-level residual structure is introduced to fuse features of different scales, realize multi-scale segmentation prediction, and improve the accuracy of image segmentation; further, edge supervision is incorporated into the training, and a jointly weighted network loss function is used to help the semantic segmentation model better learn the boundary information of the object, and enhance the semantic segmentation model's perception and learning ability of defect details in the shallow layer, thereby effectively improving the precision and accuracy of image segmentation.

[0076] Example 2

[0077] In this embodiment, a method for semantic segmentation of appearance defects of industrial products includes:

[0078] Create a dataset of industrial product appearance images to be inspected, build a semantic segmentation model, train and test the semantic segmentation model, input the images to be inspected into the semantic segmentation model, and output the semantic segmentation results of industrial product appearance defects.

[0079] The data set for preparing the appearance image of the industrial product to be detected includes:

[0080] S11, obtaining an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, comparing the abnormal sample with the normal sample image, and determining the defect type of the abnormal sample;

[0081] S12. Use an image annotation tool to perform defect annotation on abnormal samples of determined defect types to obtain abnormal defect images, and assign labels of corresponding defect types to different defect targets in the abnormal defect images;

[0082] S13. The abnormal defect images and the labels are made into a data set.

[0083] The data set is preprocessed, including:

[0084] S21. Performing data enhancement on the data set to obtain a data enhanced data set;

[0085] S22. Divide the data-enhanced dataset into a training set and a test set according to a set ratio.

[0086] The data set is preprocessed, and data quality is enhanced by adjusting image size, normalization and other data improvement methods to ensure the validity of input image data and improve the generalization ability of the model. The preprocessed data set is then divided into a set ratio (such as 8:2) and randomly divided into training sets and test sets. The SAM-based image annotation tool is used to assign corresponding type labels to different targets in the image. The original image and the annotated image are then stored in different folders, and divided into training sets and test sets, and the VOC format is used for labeling and differentiation.

[0087] Building a semantic segmentation model includes:

[0088] The ResNet-50 encoder part of the semantic segmentation model introduces a residual structure in each layer, which effectively optimizes the gradient vanishing and model degradation problems of the neural network. This improvement enables the semantic segmentation model to better learn and optimize deep features, while making it easier for gradients to propagate in the network, thereby speeding up the training of the model and improving the accuracy and generalization ability of the model.

[0089] The ECA-Net channel attention mechanism is introduced to guide the shallow features of the network to propagate to the deep layers. This mechanism obtains one-dimensional feature vectors of different scales by performing global average pooling on feature maps of different levels, and extracts features through 1D convolution operations with efficient cross-channel information acquisition capabilities. The convolution kernel size of 1D convolution affects the number of channels that need to be considered for the calculation of each weight in the attention mechanism. Based on this, the selection of the convolution kernel size of 1D convolution is particularly important, such as Figure 3 As shown in the figure, after obtaining the feature strips, the ECA attention module extracts information through convolution and selects the convolution kernel size using the following formula:

[0090] Convolution kernel size k: C represents the number of channels of input data;

[0091] Then, the Sigmoid function is used to obtain the weight of each channel and weight the feature pixels, so that the network can focus more on the channels that need attention. Finally, the multi-scale fused image feature information extracted from different levels is passed to the decoder through jump connections.

[0092] The FPN feature pyramid decoder structure is constructed, and the FPN feature pyramid decoder introduces an inter-level residual module. The present invention uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of FPN feature pyramid decoder is passed from bottom to top to the input end of the decoder of the previous layer to restore the segmentation result with higher resolution. By fusing features of different scales, multi-scale segmentation prediction is achieved to improve its accuracy.

[0093] The training model uses the coefficient loss function, the cross entropy loss function and the focus loss function as the main loss function. Based on this, the present invention introduces a new training strategy and defines a new network loss function using a fusion weighted loss function algorithm. First, the edge information of the label image is extracted using the Laplacian operator edge extraction algorithm of OpenCV, and then edge feature maps of label images of different scales are obtained through convolutions of different step lengths, which are upsampled to their original size and fused to achieve dynamic reshaping. In the present invention, the third layer of the model is selected to generate a detail feature map, and the edge detail loss function is calculated under the supervision of edge information;

[0094] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0095] L detail =L dice +L bce

[0096] Among them, L detail represents the edge detail loss function calculated for the label feature map, L dice represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0097] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0098] L main =L dice +L ce +L focal

[0099] Among them, L main Represents the main loss function calculated from the feature map of the final output, L cerepresents the cross entropy loss function, L focal represents the focal loss function;

[0100] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0101] loss = λL detail +μL main

[0102] Among them, loss represents the network loss function, λ represents the first weighted parameter, and μ represents the second weighted parameter.

[0103] Example 3

[0104] In this embodiment, a mobile phone screen data set is used as an example to explain in detail:

[0105] Obtain mobile phone screen image data, compare normal sample images with abnormal sample images, and determine the defect category of the abnormal sample.

[0106] Use image annotation tools to annotate defects on all mobile phone screen images, and create a data set with defect images and their corresponding labels.

[0107] Process the data set, preprocess the sample data, and use methods such as adjusting the image size and normalization to improve the data quality, ensure the effectiveness of the input model and improve its generalization ability, and then randomly divide the preprocessed data set into training set and test set;

[0108] The dataset contains 25,000 images of mobile phone screens with three types of defects: tin ash, bubbles, and scratches. The SAM-based image annotation tool is used to assign corresponding types to different objects in the image. The original images and annotated images are then placed in different folders and divided into training and test sets; the VOC format is used for distinction.

[0109] Equipped with a U-net-like semantic segmentation model based on edge supervision and multi-scale detail aggregation.

[0110] The present invention is implemented using Python language, and the semantic segmentation model described in step 4 is specifically:

[0111] U-net-like semantic segmentation models based on edge supervision and multi-scale detail aggregation are as follows: Figure 3As shown in the figure, the network encoder is replaced with ResNet-50, and the introduction of residual structure effectively improves the accuracy, robustness and speed of the network model segmentation algorithm. The network input data is a 512*512*3 mobile phone screen image. The image is feature extracted through a convolutional neural network, in which each layer of convolution is connected to a layer of rectified linear unit and a maximum pooling operation. Finally, after a series of bilinear interpolation upsampling and jump cascades, the image information features are aggregated and restored to the original size. The decoder part of the model adopts the FPN feature pyramid structure, and the inter-level residual module is introduced to fuse features of different scales, realizing multi-scale segmentation prediction and improving the accuracy of image segmentation. The jump cascade part introduces the ECA-Net channel attention mechanism, as shown in the figure. Figure 3 In the dotted box, the channel attention module guides the shallow features of the network to propagate to the deep layers, allowing the network to focus more on the channels that need attention.

[0112] like Figure 4 As shown, the present invention uses a training method based on edge supervision and adopts a joint weighted network loss function to train the model. The image edge information of the label image is extracted using the Laplacian operator edge extraction algorithm of OpenCV. After obtaining edge feature maps of images of different scales through convolutions of different step lengths, they are upsampled and restored to their original size and fused to achieve dynamic reshaping.

[0113] The Laplace convolution kernel is:

[0114]

[0115] In the present invention, the third layer of the model is selected to generate a detail feature map, and the edge detail loss function is calculated under the supervision of edge information;

[0116] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0117] L detail =L dice +L bce

[0118] Among them, L detail represents the edge detail loss function calculated for the label feature map, L dice represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0119] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0120] L main =L dice +L ce+L focal

[0121] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, L focal represents the focal loss function;

[0122] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0123] loss = λL detail +μL main

[0124] Among them, loss represents the network loss function, λ represents the first weighted parameter, and μ represents the second weighted parameter.

[0125] The training set data is passed into the network model for multiple iterations of training, such as Figure 2 As shown, after the model converges, a trained semantic segmentation model is obtained. The trained semantic segmentation model is the optimal network model. The trained semantic segmentation model is used to input the test set into the trained semantic segmentation model to determine that the test effect is good, such as Figure 5 As shown, the circles in the figure are not marked with defect types.

[0126] Example 4

[0127] This embodiment proposes a semantic segmentation system for appearance defects of industrial products, which is characterized by comprising:

[0128] A production module, used for producing a data set of appearance images of industrial products to be tested;

[0129] A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set;

[0130] A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function, and iteratively trains the semantic segmentation model through the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; and inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model;

[0131] The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

[0132] In this embodiment, a semantic segmentation model of industrial product appearance defects is first constructed, and then an edge supervision-based training method is used to incorporate edge supervision into the training, and a jointly weighted network loss function is used to perform multiple iterative training on the semantic segmentation model to help the semantic segmentation model better learn the boundary information of the object, and enhance the semantic segmentation model's perception and learning ability of defect details in the shallow layer, so as to obtain a trained semantic segmentation model. The test set is further input into the trained semantic segmentation model for testing to obtain the final semantic segmentation model. The final semantic segmentation model outputs the semantic segmentation results of industrial product appearance defects, effectively improving the precision and accuracy of image segmentation.

[0133] Obviously, the above embodiments of the present invention are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for semantic segmentation of appearance defects of industrial products, characterized in that: The following steps are involved: S1. Prepare a dataset of appearance images of industrial products to be tested; S2, performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set; S3, constructing a semantic segmentation model, using an edge-supervised training method, adopting a jointly weighted network loss function, and iteratively training the semantic segmentation model through the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model; S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation result of the appearance defects of industrial products.

2. The method for semantic segmentation of appearance defects of industrial products according to claim 1, characterized in that: The data set for preparing the appearance image of the industrial product to be detected includes: S11, obtaining an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, comparing the abnormal sample with the normal sample image, and determining the defect type of the abnormal sample; S12. Use an image annotation tool to perform defect annotation on abnormal samples of determined defect types to obtain abnormal defect images, and assign labels of corresponding defect types to different defect targets in the abnormal defect images; S13. The abnormal defect images and the labels are made into a data set.

3. The method for semantic segmentation of appearance defects of industrial products according to claim 1, characterized in that: The data preprocessing of the data set includes: S21. Performing data enhancement on the data set to obtain a data enhanced data set; S22. Divide the data-enhanced dataset into a training set and a test set according to a set ratio.

4. The method for semantic segmentation of appearance defects of industrial products according to claim 1, characterized in that: The semantic segmentation model includes a ResNet-50 encoder, an ECA attention module, and an FPN feature pyramid decoder connected in sequence.

5. The method for semantic segmentation of appearance defects of industrial products according to claim 4, characterized in that: The ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input into the semantic segmentation model; it includes a total of five convolutional layers, each of which is connected to a layer of rectified linear unit and a maximum pooling operation.

6. The method for semantic segmentation of appearance defects of industrial products according to claim 4, characterized in that: The ECA attention module includes a global average pooling layer, a 1D convolution layer, and a Sigmoid activation function layer; The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses the 1D convolution operation with efficient cross-channel information acquisition capability to extract feature pixels of the one-dimensional feature vector. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result. The weighted result is connected to the one-dimensional feature vector through a jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder.

7. The method for semantic segmentation of appearance defects of industrial products according to claim 4, characterized in that: The FPN feature pyramid decoder includes several layers of decoders. The FPN feature pyramid decoder uses a bilinear interpolation method to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer.

8. The method for semantic segmentation of appearance defects of industrial products according to claim 1, characterized in that: The edge-supervised training method includes: using the Laplacian operator edge extraction algorithm to extract image edge information of the label image, obtaining edge feature maps of images of different scales through convolutions of different step lengths, and then upsampling and restoring them to their original size and fusing them to achieve dynamic reshaping.

9. The method for semantic segmentation of appearance defects of industrial products according to claim 8, characterized in that: The network loss function using joint weighting includes: Calculate the edge detail loss function under the guidance of edge information supervision; The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows: L detail ＝ dice + bce Among them, L detail represents the edge detail loss function calculated for the label feature map, L dice represents the coefficient loss function, L bce represents the binary cross entropy loss function; The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows: L main ＝ dice + ce + focal Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, L focal represents the focal loss function; The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows: lpss=λL detail +L main Among them, loss represents the network loss function, λ represents the first weighted parameter, and μ represents the second weighted parameter.

10. A semantic segmentation system for industrial product appearance defects, characterized in that: include: A production module, used for producing a data set of appearance images of industrial products to be tested; A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set; A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function, and iteratively trains the semantic segmentation model through the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; and inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model; The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

Images