A method and system for semantic segmentation of industrial product appearance defects

Abstract

The present invention discloses a method and system for semantic segmentation of industrial product appearance defects, relating to the field of product defect detection technology. The method comprises preparing a dataset of industrial product appearance images to be detected; performing data preprocessing on the dataset to obtain a preprocessed dataset, and dividing the preprocessed dataset into a training set and a test set; constructing a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function, iteratively training the semantic segmentation model with the training set until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; inputting the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model; and inputting the image to be detected into the final semantic segmentation model, thereby outputting a semantic segmentation result for industrial product appearance defects. The present invention effectively improves the precision and accuracy of image segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of product defect detection, and more specifically, to a method and system for semantic segmentation of appearance defects of industrial products. Background Art

[0002] Surface defect detection is widely used in various industrial sectors, including 3C, semiconductors and electronics, automotive, chemical, pharmaceutical, light industry, and military industries. According to statistics, over 80% of industrial surface defect detection currently relies on manual inspection methods. Manufacturing companies such as Foxconn and Bern Optical employ large numbers of quality inspection workers and conduct inspections on an assembly line basis. However, manual inspection methods have drawbacks such as high labor intensity, poor detection stability and consistency, low automation, low production efficiency, and difficulty in implementing lean production, making it difficult to meet large-scale quality inspection needs. In recent years, with the emergence of new technologies in fields such as industrial imaging, computer vision, and deep learning, vision-based industrial defect detection technology has made significant progress, becoming an effective solution for product appearance quality inspection and attracting strong attention from academia and industry.

[0003] Semantic segmentation is a technology in the field of computer vision that aims to correctly classify each pixel in a digital image or video frame into a predefined category. Semantic segmentation allows different objects and regions in an image to be distinguished, and each object can be identified and located. U-Net is a deep learning model specifically designed for semantic segmentation tasks. U-Net utilizes a common convolutional neural network architecture, consisting of an encoder and a decoder. Its simple and scalable architecture has made it widely used in various application scenarios. However, its fully convolutional network structure has certain limitations. Traditional models cannot efficiently extract contextual information between multiple layers while ensuring accurate positioning. This results in insensitivity to local information and a tendency to overlook boundary details, affecting segmentation accuracy. Currently, U-Net-based semantic segmentation models have achieved remarkable results in industrial product appearance defect detection, but they still have some problems and shortcomings, such as sensitivity to noise interference, difficulty distinguishing defects from background, and difficulty handling complex boundaries. Summary of the Invention

[0004] In order to solve the problems of low precision and accuracy of image segmentation in the existing technology, a semantic segmentation method and system for industrial product appearance defects are proposed to effectively improve the precision and accuracy of image segmentation.

[0005] The technical solutions of the present invention are as follows:

[0006] The present invention discloses a method for semantic segmentation of appearance defects of industrial products, comprising the following steps:

[0007] S1. Create a dataset of appearance images of industrial products to be tested;

[0008] S2. performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set;

[0009] S3. Construct a semantic segmentation model, use an edge-supervised training method, adopt a jointly weighted network loss function, and iteratively train the semantic segmentation model on the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; input the trained semantic segmentation model into the test set for testing, thereby obtaining a final semantic segmentation model;

[0010] S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation results of industrial product appearance defects.

[0011] Preferably, the data set for producing the appearance image of the industrial product to be inspected includes:

[0012] S11. Acquire an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, compare the abnormal sample with the normal sample image, and determine the defect type of the abnormal sample;

[0013] S12. Use an image annotation tool to annotate the abnormal samples of the determined defect type to obtain an abnormal defect image, and assign labels of corresponding defect types to different defect targets in the abnormal defect image;

[0014] S13. Create a data set with the abnormal defect images and the labels.

[0015] Preferably, the data preprocessing of the data set includes:

[0016] S21. Performing data enhancement on the data set to obtain a data-enhanced data set;

[0017] S22. Divide the data-augmented dataset into a training set and a test set according to a set ratio.

[0018] Preferably, the semantic segmentation model includes a ResNet-50 encoder, an ECA attention module and an FPN feature pyramid decoder connected in sequence.

[0019] Preferably, the ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input into the semantic segmentation model; it includes a total of five convolutional layers, each of which is connected to a layer of rectified linear unit and a maximum pooling operation.

[0020] Preferably, the ECA attention module includes a global average pooling layer, a 1D convolution layer and a Sigmoid activation function layer;

[0021] The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses the 1D convolution operation with efficient cross-channel information acquisition capability to extract feature pixels of the one-dimensional feature vector. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result. The weighted result is connected to the one-dimensional feature vector through a jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder.

[0022] Preferably, the FPN feature pyramid decoder includes several layers of decoders, and the FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top, and the output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer.

[0023] Preferably, the edge-supervised training method includes: using the Laplacian operator edge extraction algorithm to extract the image edge information of the label map, obtaining image edge feature maps of different scales through convolutions of different step lengths, upsampling them to their original size and fusing them to achieve dynamic reshaping.

[0024] Preferably, the network loss function using joint weighting includes:

[0025] Calculate the edge detail loss function under the guidance of edge information supervision;

[0026] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0027] L detail =L dice +L nce

[0028] Among them, L detail Represents the edge detail loss function calculated for the label feature map, L dice Represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0029] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0030] L main =L dice +L ce +L focal

[0031] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, L focal represents the focal loss function;

[0032] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0033] loss = λL detail +μL main

[0034] Among them, loss represents the network loss function, λ represents the first weighting parameter, and μ represents the second weighting parameter.

[0035] The present invention also proposes a semantic segmentation system for industrial product appearance defects, comprising:

[0036] A production module, used to produce a dataset of appearance images of industrial products to be inspected;

[0037] A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set;

[0038] A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function. The semantic segmentation model is iteratively trained multiple times using the training set until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model. The test set is input into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model.

[0039] The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

[0040] Beneficial effects of the present invention:

[0041] The present invention proposes a method and system for semantic segmentation of industrial product appearance defects. First, a semantic segmentation model of industrial product appearance defects is constructed. Then, a training method based on edge supervision is used to incorporate edge supervision into the training. A jointly weighted network loss function is adopted to perform multiple iterative training on the semantic segmentation model to help the semantic segmentation model better learn the boundary information of objects, improve the semantic segmentation model's perception and learning ability of defect details in the shallow layer, and obtain a trained semantic segmentation model. The test set is further input into the trained semantic segmentation model for testing to obtain a final semantic segmentation model. The final semantic segmentation model outputs the semantic segmentation results of industrial product appearance defects, effectively improving the precision and accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a method for semantic segmentation of appearance defects of industrial products provided in an embodiment.

[0043] Figure 2 A schematic diagram of the specific process of the semantic segmentation method for industrial product appearance defects provided in the embodiment.

[0044] Figure 3 A schematic diagram of a semantic segmentation model provided in an embodiment.

[0045] Figure 4 A schematic diagram of an edge-supervised training method provided in an embodiment.

[0046] Figure 5 An example diagram of the semantic segmentation results provided in the embodiment.

[0047] Figure 6 A schematic diagram of the structure of the industrial product appearance defect semantic segmentation system provided in the embodiment. DETAILED DESCRIPTION

[0048] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] In this embodiment, see Figure 1 and Figure 2 ,A semantic segmentation method for appearance defects of industrial products, comprising the following steps:

[0051] S1. Create a dataset of appearance images of industrial products to be tested;

[0052] S2. performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set;

[0053] S3. Construct a semantic segmentation model, use an edge-supervised training method, adopt a jointly weighted network loss function, and iteratively train the semantic segmentation model multiple times through the training set until the network loss function converges to obtain a trained semantic segmentation model; input the test set into the trained semantic segmentation model for testing to obtain the final semantic segmentation model; wherein, the semantic segmentation model includes a ResNet-50 encoder, an ECA attention module, and an FPN feature pyramid decoder connected in sequence.

[0054] S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation results of industrial product appearance defects.

[0055] In this embodiment, the ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input into the semantic segmentation model; it includes a total of five convolutional layers, each of which is connected to a layer of rectified linear unit and a maximum pooling operation.

[0056] More specifically, the ECA attention module can efficiently obtain the correlation information between channels, which is used to guide the shallow features of the network to propagate to the deep layers, thereby improving the model's ability to process local and global information. The ECA attention module includes a global average pooling layer, a 1D convolution layer, and a Sigmoid activation function layer.

[0057] The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses the 1D convolution operation with efficient cross-channel information acquisition capability to extract the feature pixels of the one-dimensional feature vector. The convolution kernel size of the 1D convolution affects the number of channels for calculating each weight in the attention mechanism. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result, so that the network can focus more on the channels that need attention. The weighted result is connected to the one-dimensional feature vector through jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder.

[0058] More specifically, the convolution kernel size k for 1D convolution is selected as follows:

[0059] Convolution kernel size k: C represents the number of channels of input data;

[0060] More specifically, the FPN feature pyramid decoder includes several layers of decoders. The FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer. By fusing features of different scales, multi-scale segmentation prediction is achieved.

[0061] More specifically, the image features extracted by the ResNet-50 encoder are aggregated and restored to their original size through upsampling by the FPN feature pyramid decoder and skip-concatenation of the ECA attention module to facilitate visualization of the segmentation prediction results. By aggregating semantic features at different scales, the model can effectively capture a wider range of global information and improve the global receptive field.

[0062] In this embodiment, the edge-supervised training method includes: using OpenCV's Laplacian operator edge extraction algorithm to extract image edge information of the label image, obtaining image edge feature maps of different scales through convolution with different step lengths, upsampling them to their original size, and fusing them to achieve dynamic reshaping.

[0063] More specifically, the network loss function using joint weighting includes:

[0064] Calculate the edge detail loss function under the guidance of edge information supervision;

[0065] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0066] L detail =L dice +L bce

[0067] Among them, L detail Represents the edge detail loss function calculated for the label feature map, L dice Represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0068] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0069] L main =L dice +L ce +L focal

[0070] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, Lfocal represents the focal loss function;

[0071] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0072] loss = λL detail +μL main

[0073] Among them, loss represents the network loss function, λ represents the first weighting parameter, and μ represents the second weighting parameter.

[0074] In this embodiment, the test set is input into the trained semantic segmentation model for testing. If the defect type of the output semantic segmentation result is correct and different defect types are marked with different colors in the figure, the test is successful; otherwise, the test fails and iterative training continues.

[0075] In this embodiment, first, the ResNet-50 encoder is used to optimize the problems of gradient vanishing and model degradation in the neural network, which speeds up the network training speed, improves the accuracy of image segmentation and the generalization ability of the semantic segmentation model; secondly, by introducing the channel attention mechanism of the ECA attention module, the network is made more focused on the channels that need attention; then, through the FPN feature pyramid decoder, the inter-level residual structure is introduced to fuse features of different scales, realize multi-scale segmentation prediction, and improve the accuracy of image segmentation; further, edge supervision is incorporated into the training, and a jointly weighted network loss function is adopted to help the semantic segmentation model better learn the boundary information of the object, enhance the semantic segmentation model's perception and learning ability of defect details in the shallow layer, thereby effectively improving the precision and accuracy of image segmentation.

[0076] Example 2

[0077] In this embodiment, a method for semantic segmentation of appearance defects of industrial products includes:

[0078] Create a dataset of appearance images of industrial products to be inspected, build a semantic segmentation model, train and test the semantic segmentation model, input the images to be inspected into the semantic segmentation model, and output the semantic segmentation results of industrial product appearance defects.

[0079] The data set for producing the appearance image of the industrial product to be inspected includes:

[0080] S11. Acquire an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, compare the abnormal sample with the normal sample image, and determine the defect type of the abnormal sample;

[0081] S12. Use an image annotation tool to annotate the abnormal samples of the determined defect type to obtain an abnormal defect image, and assign labels of corresponding defect types to different defect targets in the abnormal defect image;

[0082] S13. Create a data set with the abnormal defect images and the labels.

[0083] Performing data preprocessing on the data set includes:

[0084] S21. Performing data enhancement on the data set to obtain a data-enhanced data set;

[0085] S22. Divide the data-augmented dataset into a training set and a test set according to a set ratio.

[0086] The dataset was preprocessed, and data quality was enhanced using methods such as image resizing and normalization to ensure the validity of the input image data and improve the generalization ability of the model. The preprocessed dataset was then randomly divided into training and test sets with a set ratio (e.g., 8:2). A SAM-based image annotation tool was used to assign corresponding type labels to different objects in the image. The original images and annotated images were then stored in different folders and divided into training and test sets, using the VOC format for labeling and differentiation.

[0087] Building a semantic segmentation model involves:

[0088] The ResNet-50 encoder part of the semantic segmentation model introduces a residual structure in each layer, effectively optimizing the problems of vanishing gradients and model degradation in neural networks. This improvement enables the semantic segmentation model to better learn and optimize deep features, while making it easier for gradients to propagate in the network, thereby speeding up model training and improving model accuracy and generalization capabilities.

[0089] The ECA-Net channel attention mechanism is introduced to guide the shallow features of the network to propagate to the deep layers. This mechanism performs global average pooling on feature maps of different layers to obtain one-dimensional feature vectors of different scales, and extracts features through 1D convolution operations with efficient cross-channel information acquisition capabilities. The size of the convolution kernel of 1D convolution affects the number of channels that need to be considered for the calculation of each weight in the attention mechanism. Based on this, the selection of the convolution kernel size of 1D convolution is particularly important, such as Figure 3 As shown in the figure, after obtaining the feature strips, the ECA attention module extracts information through convolution and selects the convolution kernel size using the following formula:

[0090] Convolution kernel size k: C represents the number of channels of input data;

[0091] Then, a Sigmoid function is used to obtain the weight of each channel and weight the feature pixels, allowing the network to focus more on the channels that need attention. Finally, the multi-scale fused image feature information extracted from different levels is passed to the decoder through skip connections.

[0092] A FPN feature pyramid decoder architecture is constructed, which incorporates an inter-level residual module. This method uses bilinear interpolation to transfer and aggregate semantic features at different scales from bottom to top. The output of each FPN feature pyramid decoder layer is fed from bottom to top into the input of the decoder at the previous layer to recover a higher-resolution segmentation result. By fusing features at different scales, multi-scale segmentation prediction is achieved, improving its accuracy.

[0093] The training model uses the coefficient loss function, cross entropy loss function and focus loss function as the main loss function. Based on this foundation, the present invention introduces a new training strategy and defines a new network loss function using a fusion weighted loss function algorithm. First, the edge information of the label image is extracted using OpenCV's Laplacian operator edge extraction algorithm. Then, edge feature maps of label images of different scales are obtained through convolutions of different step lengths. They are upsampled to their original size and fused to achieve dynamic reshaping. In the present invention, the third layer of the model is selected to generate a detail feature map, and the edge detail loss function is calculated under the supervision of edge information.

[0094] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0095] L detail =L dice +L bce

[0096] Among them, L detail Represents the edge detail loss function calculated for the label feature map, L dice Represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0097] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0098] L main =L dice +L ce +L focal

[0099] Among them, L main Represents the main loss function calculated from the feature map of the final output, L cerepresents the cross entropy loss function, L focal represents the focal loss function;

[0100] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0101] loss = λL detail +μL main

[0102] Among them, loss represents the network loss function, λ represents the first weighting parameter, and μ represents the second weighting parameter.

[0103] Example 3

[0104] In this embodiment, a mobile phone screen dataset is used as an example to illustrate:

[0105] Obtain mobile phone screen image data, compare normal sample images with abnormal sample images, and determine the defect category of the abnormal sample.

[0106] Use image annotation tools to annotate defects on all mobile phone screen images, and create a dataset of defect images and their corresponding labels.

[0107] Process the dataset, preprocess the sample data, and improve the data quality by adjusting the image size and normalization to ensure the validity of the input model and improve its generalization ability. Then, randomly divide the preprocessed dataset into training and test sets;

[0108] The dataset contains 25,000 images of mobile phone screens, each displaying three types of defects: tin dust, bubbles, and scratches. Using a SAM-based image annotation tool, we assigned corresponding types to different objects in the images. The original and annotated images were then placed in separate folders and divided into training and test sets, using the VOC format for classification.

[0109] Equipped with a U-net-like semantic segmentation model based on edge supervision and multi-scale detail aggregation.

[0110] The present invention is implemented using Python language. The semantic segmentation model described in step 4 is specifically as follows:

[0111] U-net-like semantic segmentation models based on edge supervision and multi-scale detail aggregation are as follows: Figure 3As shown in the figure, the network encoder is replaced with ResNet-50, and the introduction of residual structure effectively improves the accuracy, robustness and speed of the network model segmentation algorithm. The network input data is a 512*512*3 mobile phone screen image. The image feature is extracted through a convolutional neural network, in which each layer of convolution is connected to a layer of rectified linear unit and a maximum pooling operation. Finally, after a series of bilinear interpolation upsampling and jump cascade, the image information features are aggregated and restored to the original size. The decoder part of the model adopts the FPN feature pyramid structure, and the inter-level residual module is introduced to fuse features of different scales, realize multi-scale segmentation prediction, and improve the accuracy of image segmentation. The jump cascade part introduces the ECA-Net channel attention mechanism, as shown in the figure. Figure 3 In the dotted box, the channel attention module guides the shallow features of the network to propagate to the deep layers, allowing the network to focus more on the channels that need attention.

[0112] like Figure 4 As shown, the present invention uses an edge-supervised training method and a jointly weighted network loss function to train the model. The Laplacian edge extraction algorithm of OpenCV is used to extract the image edge information of the label map. Convolutions of different step sizes are used to obtain edge feature maps of images of different scales. These maps are then upsampled to their original size and fused to achieve dynamic reshaping.

[0113] The Laplace convolution kernel is:

[0114]

[0115] In this invention, the third layer of the model is selected to generate the detail feature map, and the edge detail loss function is calculated under the guidance of edge information supervision;

[0116] The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows:

[0117] L detail =L dice +L bce

[0118] Among them, L detail Represents the edge detail loss function calculated for the label feature map, L dice Represents the coefficient loss function, L bce represents the binary cross entropy loss function;

[0119] The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows:

[0120] L main =L dice +L ce+L focal

[0121] Among them, L main Represents the main loss function calculated from the feature map of the final output, L ce represents the cross entropy loss function, L focal represents the focal loss function;

[0122] The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows:

[0123] loss = λL detail +μL main

[0124] Among them, loss represents the network loss function, λ represents the first weighting parameter, and μ represents the second weighting parameter.

[0125] The training set data is passed into the network model for multiple iterative training, such as Figure 2 As shown, after the model converges, a trained semantic segmentation model is obtained. The trained semantic segmentation model is the optimal network model. Using the trained semantic segmentation model, the test set is input into the trained semantic segmentation model to determine whether the test effect is good, as shown in FIG. Figure 5 As shown, the circles in the figure are not marked with defect types.

[0126] Example 4

[0127] This embodiment proposes a semantic segmentation system for industrial product appearance defects, which includes:

[0128] A production module, used to produce a dataset of appearance images of industrial products to be inspected;

[0129] A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set;

[0130] A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function. The semantic segmentation model is iteratively trained multiple times using the training set until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model. The test set is input into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model.

[0131] The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

[0132] In this embodiment, a semantic segmentation model of industrial product appearance defects is first constructed, and then a training method based on edge supervision is used to incorporate edge supervision into the training, and a jointly weighted network loss function is adopted to perform multiple iterative training on the semantic segmentation model to help the semantic segmentation model better learn the boundary information of the object, and improve the semantic segmentation model's perception and learning ability of defect details in the shallow layer, so as to obtain a trained semantic segmentation model. The test set is further input into the trained semantic segmentation model for testing to obtain the final semantic segmentation model. The final semantic segmentation model outputs the semantic segmentation results of industrial product appearance defects, effectively improving the precision and accuracy of image segmentation.

[0133] Obviously, the above embodiments of the present invention are merely examples for the purpose of illustrating the present invention, and are not intended to limit the implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for semantic segmentation of industrial product appearance defects, characterized by: The following steps are involved: S1. Create a dataset of appearance images of industrial products to be tested; S2. performing data preprocessing on the data set to obtain a preprocessed data set, and dividing the preprocessed data set into a training set and a test set; S3. Construct a semantic segmentation model, use an edge-supervised training method, adopt a joint weighted network loss function, and iteratively train the semantic segmentation model on the training set for multiple times until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model; input the test set into the trained semantic segmentation model for testing, thereby obtaining a final semantic segmentation model; the semantic segmentation model includes a ResNet-50 encoder, an ECA attention module, and an FPN feature pyramid decoder connected in sequence; The ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input to the semantic segmentation model; it includes five convolutional layers, each of which is connected to a rectified linear unit layer and a maximum pooling operation; the ECA attention module includes a global average pooling layer, a 1D convolution layer, and a sigmoid activation function layer; The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses a 1D convolution operation with efficient cross-channel information acquisition capability to extract feature pixels of the one-dimensional feature vector. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result. The weighted result is connected to the one-dimensional feature vector through a jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder. The FPN feature pyramid decoder includes several layers of decoders. The FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer. S4. Input the image to be detected into the final semantic segmentation model and output the semantic segmentation results of industrial product appearance defects.

2. The method for semantic segmentation of industrial product appearance defects according to claim 1, characterized in that: The data set for producing the appearance image of the industrial product to be inspected includes: S11. Acquire an appearance image of an industrial product to be inspected, wherein the appearance image of the industrial product to be inspected includes an abnormal sample and a normal sample, compare the abnormal sample with the normal sample image, and determine the defect type of the abnormal sample; S12. Use an image annotation tool to annotate the abnormal samples of the determined defect type to obtain an abnormal defect image, and assign labels of corresponding defect types to different defect targets in the abnormal defect image; S13. Create a data set with the abnormal defect images and the labels.

3. The method for semantic segmentation of industrial product appearance defects according to claim 1, characterized in that: The performing data preprocessing on the data set includes: S21. Performing data enhancement on the data set to obtain a data-enhanced data set; S22. Divide the data-augmented dataset into a training set and a test set according to a set ratio.

4. The method for semantic segmentation of appearance defects of industrial products according to claim 1, characterized in that: The edge-supervised training method includes: using the Laplacian operator edge extraction algorithm to extract image edge information of the label map, obtaining edge feature maps of images of different scales through convolutions of different step lengths, and then upsampling them to restore them to their original size and fusing them to achieve dynamic reshaping.

5. The method for semantic segmentation of industrial product appearance defects according to claim 4, characterized in that: The network loss function using joint weighting includes: Calculate the edge detail loss function under the guidance of edge information supervision; The calculation of the edge detail loss function is optimized by using the coefficient loss function and the binary cross entropy loss function as follows: in, Represents the edge detail loss function calculated for the label feature map, represents the coefficient loss function, represents the binary cross entropy loss function; The coefficient loss function, cross entropy loss function and focus loss function are selected as the main loss functions. The calculation formula of the main loss function is as follows: in, Represents the main loss function calculated from the feature map of the final output, represents the cross entropy loss function, represents the focal loss function; The main loss function and the edge detail loss function are jointly weighted to form a network loss function, and the calculation formula of the network loss function is as follows: in, represents the network loss function, represents the first weighting parameter, represents the second weighting parameter.

6. A semantic segmentation system for industrial product appearance defects, characterized by: include: A production module, used to produce a dataset of appearance images of industrial products to be inspected; A preprocessing module is used to perform data preprocessing on the data set to obtain a preprocessed data set, and divide the preprocessed data set into a training set and a test set; A semantic segmentation model construction module is used to construct a semantic segmentation model, using an edge-supervised training method and a jointly weighted network loss function. The semantic segmentation model is iteratively trained multiple times on the training set until the network loss function reaches convergence, thereby obtaining a trained semantic segmentation model. The test set is input into the trained semantic segmentation model for testing to obtain a final semantic segmentation model. The semantic segmentation model includes a ResNet-50 encoder, an ECA attention module, and an FPN feature pyramid decoder connected in sequence. The ResNet-50 encoder is used to perform feature extraction and maximum pooling downsampling on the image input to the semantic segmentation model; it includes five convolutional layers, each of which is connected to a rectified linear unit layer and a maximum pooling operation; the ECA attention module includes a global average pooling layer, a 1D convolution layer, and a sigmoid activation function layer; The global average pooling layer performs global average pooling on the feature map input to the ECA attention module to obtain one-dimensional feature vectors of different scales. The 1D convolution layer uses a 1D convolution operation with efficient cross-channel information acquisition capability to extract feature pixels of the one-dimensional feature vector. The Sigmoid activation function layer weights the feature pixels to obtain a weighted result. The weighted result is connected to the one-dimensional feature vector through a jump connection to obtain multi-scale fused image feature information of different levels, and the image feature information is passed to the FPN feature pyramid decoder. The FPN feature pyramid decoder includes several layers of decoders. The FPN feature pyramid decoder uses bilinear interpolation to transfer and aggregate semantic features of different scales from bottom to top. The output of each layer of decoder is passed from bottom to top to the input end of the decoder of the previous layer. The output module inputs the image to be detected into the final semantic segmentation model and outputs the semantic segmentation results of industrial product appearance defects.

Images