Powder spreading defect detection method based on deep learning image segmentation

Abstract

The invention relates to a powder spreading defect detection method based on deep learning image segmentation. According to the powder spreading defect detection method, data enhancement and training sample expansion are performed on the image data of the marking training set, more samples are generated from limited original data, the diversity of the data is increased, the problem of insufficient data volume is solved, and the generalization of an image segmentation network model is improved. A final image segmentation network model of lightweight deep learning is obtained through establishment of a primary image segmentation model, multi-round model training and training effect verification and evaluation, and defects such as insufficient powder spreading, fine defects (warping) and slender defects (scraper scratches) in the powder spreading process can be effectively detected by using the final image segmentation network model. And the whole process does not need to manually adjust parameters in real time due to scene change, so that the operation is simple and convenient, and the false detection rate is reduced. By using the powder spreading defect detection method based on deep learning, the reliability and accuracy of powder spreading defect detection are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of additive manufacturing, and in particular to a powder spreading defect detection method based on deep learning image segmentation. Background Art

[0002] Laser selective melting technology is a new additive manufacturing technology. It has the advantage of being able to manufacture complex structural parts in a short period of time and has been widely used. However, there are some problems in the process of laser selective melting technology that need to be optimized. For example, the powder spreading process is an important link in the laser selective melting forming process, and the quality of the powder spreading directly determines the quality of the molded parts. The powder spreading process often has defects such as insufficient powder spreading, scraper scratches and warping, which may cause the final printed parts to be scrapped. Therefore, during the part processing process, it is necessary to check the powder spreading quality during the powder spreading process to ensure the final quality of the parts.

[0003] Most powder spreading detection methods are based on image processing and image feature fusion methods to detect defect edges. These methods have the advantages of fast calculation speed and low computing resource consumption, and have good defect detection effects in a single scenario. However, during the laser melting powder spreading process, various interference factors such as insufficient powder supply, scraper scratches, and warping often appear on the powder bed surface. The manifestation of interference varies with the processing material, and the scene is relatively complex. These methods based on image processing or image feature fusion often require manual and frequent parameter adjustment to cope with these changing scenarios, and the parameters are difficult to adjust in real time. The adjustment process is cumbersome, the types of defects that can be detected are single, the false detection rate is high, and it is difficult to ensure the quality of powder spreading. Target detection methods based on deep learning are prone to miss small (warping) and slender defects (scraper scratches), affecting production efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a powder spreading defect detection method based on deep learning segmentation that can cope with various detection scenarios without frequent parameter adjustment during the detection process, and can improve the accuracy and reliability of powder spreading defect detection.

[0005] A powder spreading defect detection method based on deep learning image segmentation comprises the following steps:

[0006] Collect the powder spreading image data after the current powder spreading;

[0007] Performing defect annotation on the powder spreading image data to obtain annotated image data;

[0008] Dividing the labeled image data into a labeled training set, a labeled verification set, and a labeled test set;

[0009] Performing data augmentation on the labeled training set to obtain an expanded training set;

[0010] Build a primary image segmentation network model;

[0011] Using the labeled training set and the expanded training set as input data to perform model training on the image segmentation network model to obtain a training image segmentation network model;

[0012] Inputting the labeled verification set into the training image segmentation network model, evaluating the training effect of the training image segmentation network model, and obtaining an evaluation result; the evaluation result includes a loss function value, an average intersection-over-union ratio, and an average pixel accuracy rate;

[0013] The network model whose evaluation result meets the preset evaluation conditions is used as the final image segmentation network model;

[0014] Inputting the labeled test set into the final image segmentation network model to output a powder spreading defect result;

[0015] Among them, the preset evaluation condition is at least one of the following conditions: the loss function value gradually decreases and tends to be stable with the increase of the number of training rounds; the average intersection-over-union ratio and the average pixel accuracy gradually increase with the increase of the number of training rounds, and after the labeled verification set is input, the improvement of the average intersection-over-union ratio and the average pixel accuracy is not obvious.

[0016] In the above-mentioned powdering defect detection method based on deep learning image segmentation, in practical applications, the collection of powdering defect image data is affected by factors such as time and cost, and it is often difficult to obtain a large amount of powdering image data. In the above-mentioned powdering defect detection method, by performing data enhancement on the image data of the labeled training set to expand the training samples and obtain the expanded training set, more samples can be generated from the limited original data, increasing the diversity of the data, alleviating the problem of insufficient data volume, and improving the generalization of the image segmentation network model. At the same time, the constructed primary image segmentation model is trained for multiple rounds using the labeled training set after defect annotation, and the training effect of the model is verified and evaluated using the labeled verification set after defect annotation after each round of training, until the evaluation results show that the model has reached a convergence state, so as to obtain the final image segmentation network model of lightweight deep learning. The final image segmentation network model can be used to effectively detect defects such as insufficient powdering, small (warping), and slender defects (scraper scratches) in the powdering process, and the entire process does not require manual real-time adjustment of parameters due to scene changes, and the operation is simple, which greatly reduces the false detection rate. Therefore, the use of the above-mentioned powder spreading defect detection method based on deep learning effectively improves the reliability and accuracy of powder spreading defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Moreover, the same reference numerals are used throughout the drawings to represent the same components. In the drawings:

[0018] Figure 1 It is a flow chart of a powder spreading defect detection method based on deep learning image segmentation in a preferred embodiment of the present invention;

[0019] Figure 2 for Figure 1 A schematic flow chart of step S101 and step S102 after step S90 in the powder spreading defect detection method based on deep learning image segmentation is shown;

[0020] Figure 3 for Figure 1 The flowchart of step S40 in the powder spreading defect detection method based on deep learning image segmentation is shown;

[0021] Figure 4 for Figure 1 The flowchart of step S50 in the powder spreading defect detection method based on deep learning image segmentation is shown;

[0022] Figure 5 for Figure 4 The flowchart of step S502 in step S50 is shown;

[0023] Figure 6 for Figure 4 The flowchart of step S503 in step S50 is shown;

[0024] Figure 7 for Figure 4 The flowchart of step S504 in step S50 is shown;

[0025] Figure 8 for Figure 1 A schematic flow chart of step S90 in the powder spreading defect detection method based on deep learning image segmentation is shown. DETAILED DESCRIPTION

[0026] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0028] When describing positional relationships, unless otherwise specified, when an element is referred to as being "on" another element, it can be directly on the other element or there can be an intermediate element. It is also understood that when an element is referred to as being "between" two elements, it can be the only one between the two elements, or there can be one or more intermediate elements.

[0029] In the case of using “including”, “having”, and “comprising” described herein, another component may be added unless a clear limiting term such as “only”, “consisting of”, etc. is used. Unless mentioned otherwise, a term in the singular form may include a plural form and should not be understood as being one in number.

[0030] See also Figure 1 The powder spreading defect detection method based on the deep learning image segmentation network model in the preferred embodiment of the present invention includes steps S10 to S90.

[0031] Step S10, collecting the current powder spreading image data.

[0032] Step S20, marking defects on the powder spreading image data to obtain marked image data.

[0033] Specifically, different defect type areas in the powder spreading image data are labeled using a labeling tool to obtain labeled image data. The labeling tool may be Labelme, etc., and different defect type areas in each image in the powder spreading image data are labeled using the labeling tool to obtain a labeling file corresponding to each image.

[0034] It should be noted that the defect types in the powder spreading image include insufficient powder supply, scraper scratches, and warping.

[0035] Step S30, dividing the labeled image data into a labeled training set, a labeled verification set, and a labeled test set.

[0036] Specifically, the labeled image data is divided into a labeled training set, a labeled verification set, and a labeled test set in a ratio of 7:2:1. Of course, in other embodiments, the division ratio of the labeled training set, the labeled verification set, and the labeled test set can also be other ratios, such as 7:1.5:1.5, etc., as long as the proportion of the labeled training set is much greater than the proportion of the labeled verification set and the proportion of the labeled test set.

[0037] Step S40, performing data enhancement on the labeled training set to obtain an expanded training set.

[0038] Step S50, constructing a primary image segmentation network model.

[0039] Step S60, using the labeled training set and the expanded training set as input data to perform model training on the primary image segmentation network model to obtain a training image segmentation network model.

[0040] Specifically, the model training process adopts the stochastic gradient descent (SGD) optimization strategy to set the parameters of the primary image segmentation network model, such as the initial learning rate is 0.001, the weight decay of the learning rate is 0.0001, the batch training size is 16, the maximum number of training rounds is 12000, etc.; after the parameters are set, the images and corresponding annotation files in the labeled training set and the expanded training set are randomly read and input into the primary image segmentation network model for model training.

[0041] Step S70, input the labeled verification set into the training image segmentation network model, evaluate the training effect of the training image segmentation network model, and obtain an evaluation result. The evaluation result includes the loss function value, the average intersection-over-union ratio, and the average pixel accuracy.

[0042] Step S80, the training image segmentation network model whose evaluation results meet the preset evaluation conditions is used as the final image segmentation network model. The preset evaluation conditions are at least one of the following conditions: the loss function value gradually decreases and tends to be stable with the increase of the number of training rounds; the average intersection-over-union ratio and the average pixel accuracy gradually increase with the increase of the number of training rounds, and the average intersection-over-union ratio and the average pixel accuracy do not increase significantly after the input of the labeled verification set.

[0043] Step S90, inputting the labeled test set into the final image segmentation network model to output the powder spreading defect result.

[0044] By executing steps S10 to S90, various defects that occur during the powder spreading process can be effectively detected. In practical applications, the collection of powder spreading defect image data is affected by factors such as time and cost, and it is often difficult to obtain a large amount of powder spreading image data. By executing steps S10 to S40 to expand the training samples and obtain an expanded training set, more samples can be generated from limited original data, increasing the diversity of data, alleviating the problem of insufficient data volume, and improving the generalization of the image segmentation network model.

[0045] By executing S50 to step S80, a primary image segmentation network model is built, and the labeled training set after defect annotation is used to perform multiple rounds of model training on the primary image segmentation network model, and after each round of training, the labeled verification set after defect annotation is used to verify and evaluate the training effect of the model. When the evaluation result meets at least one of the following conditions, that is, the loss function value gradually decreases and tends to be stable with the increase in the number of training rounds, and in the later stage of training, the loss function value fluctuates within a very small range and is relatively stable; the average intersection-over-union ratio and the average pixel accuracy gradually increase with the increase in the number of training rounds, and the higher the average intersection-over-union ratio and the average pixel accuracy, the better the model splitting effect, and in the later stage of training, when the average intersection-over-union ratio and the average pixel accuracy are no longer significantly improved, it means that the model has reached a convergence state, so as to obtain the final image segmentation network model of lightweight deep learning.

[0046] By executing step S90, the above-mentioned final image segmentation network model can be used to effectively detect defects such as insufficient powder spreading, small defects (warping), and slender defects (scraper scratches) in the powder spreading process, and the entire process does not require manual real-time adjustment of parameters due to scene changes. The operation is simple and the false detection rate is greatly reduced. Therefore, the use of the above-mentioned powder spreading defect detection method based on deep learning effectively improves the reliability and accuracy of powder spreading defect detection.

[0047] Please also read Figure 2 In some embodiments, after step S90, steps S101 and S102 are further included:

[0048] Step S101, performing image processing on the powder spreading defect results to calculate the length, width and area of ​​each defect.

[0049] Step S102, judging whether the defect is within a preset control range according to the length, width and area of ​​the defect.

[0050] Therefore, by executing steps S101 to S102, the powder laying defect results obtained by defect detection through the final image segmentation network model can be secondary processed and screened to determine whether the defects occurring in the powder laying process are within the control range, thereby further improving the accuracy and reliability of powder laying defect detection.

[0051] Please also read Figure 3 In some embodiments, step S40 includes step S41 and step S42:

[0052] Step S41, establishing an image processing method set.

[0053] Step S42, using the image processing method set to perform image processing on the labeled training set for a preset number of times to obtain an expanded training set.

[0054] Among them, the processing categories of the image processing method set include image geometric transformation, image color transformation, image smoothing, image noise addition and image fusion. The processing methods of image geometric transformation include horizontal flipping, vertical flipping, rotation, translation, cropping, scaling, and affine transformation. The processing methods of image color transformation include brightness, contrast, RGB translation, hue saturation value, color equalization, hue separation, and random grid shuffling. The processing methods of image smoothing include median filtering, Gaussian filtering, motion blur, and random smoothing. The processing methods of image noise addition include Gaussian noise, salt and pepper noise, camera sensor noise, gamma noise, and fog addition.

[0055] When establishing an image processing method set, set the parameter t value, indicating that the image processing method of the tth category is selected to perform image processing, 0<t≤4; set the parameter p1, indicating the probability value of the four categories of image processing methods being randomly selected for execution; set the parameter p2, indicating the probability value of each processing method in a certain category of image processing methods being randomly selected for execution.

[0056] It should be noted that the values ​​of t correspond one-to-one to image geometric transformation, image color transformation, image smoothing, and image noise, that is, when t=1, it means that image geometric transformation is selected, when t=2, it means that image color transformation is selected..., of course, t=4 can also indicate that image geometric transformation is selected in the data enhancement processing process, t=3 indicates that image color transformation is selected..., here, there is no limitation on the t values ​​corresponding to image geometric transformation, image color transformation, image smoothing, and image noise, as long as each image processing category corresponds to a different t value.

[0057] Thus, by executing step S41, an image processing method set can be established, and various parameters can be set in the process of establishing the image processing method set. By executing step S42, the image processing method set can be used to perform a preset number of image processing on the labeled training set, and according to the various parameters set in step S41, a complex, diverse and complete image processing process on the labeled training set can be completed, thereby obtaining an expanded training set with diverse data and sufficient data volume.

[0058] In some embodiments, the primary image segmentation network model includes an image encoding module and an image decoding module. The image encoding module includes an input layer, a Stage 1 layer, a Stage 2 layer, a Stage 3 layer, a Stage 4 layer, and a Stage 5 layer. The image decoding module includes a PPM module (pyramid pooling module), an ARM module (attention refinement module), and a FFM module (feature fusion module).

[0059] Among them, the Stage1 layer and the Stage2 layer each include a ConvX module with a convolution kernel size of 3×3, and the ConvX module includes a convolution layer, a BN layer and a ReLU activation layer. The Stage3 layer, the Stage4 layer and the Stage5 layer each include 1 STDC1 module and 3 STDC2 modules. The STDC1 module includes n Block blocks (n≥2), and the STDC2 module includes n Block blocks (n≥2). Among them, the number of Block blocks n in the STDC1 module and the number of Block blocks n in the STDC2 module can be different or the same.

[0060] Furthermore, in some embodiments, in the STDC1 module, assuming that the number of output channels of the STDC1 module is N, the number of channels of the output feature maps of the n Block blocks is N / 2 respectively. i , (i=1,2,3,……,n-1), and the number of channels of the output feature map of the nth Block is the same as the number of channels of the output feature map of the n-1th Block. In the STDC2 module, assuming that the number of output channels of the STDC2 module is N, the number of channels of the output feature maps of the n Blocks is N / 2 respectively. i , (i=1,2,3,…,n-1), and the number of channels of the output feature map of the nth Block is the same as the number of channels of the output feature map of the n-1th Block.

[0061] Please also read Figure 4 , step S50 includes steps S51 to S501:

[0062] Step S51: downsample the input image by stride=2 at the Stage 1 layer to output a feature map F with a downsampling ratio of 1 / 2. Stage1The input image may be the image in step S10 and step S30, or may be some other test images provided separately, as long as the construction of the image segmentation network model can be realized.

[0063] Step S52: The feature map F Stage1 Perform a downsampling of s tride = 2 at the Stage 2 layer to output a feature map F with a downsampling ratio of 1 / 4 Stage2 .

[0064] Step S53: transform the feature map F Stage2 In the Stage 3 layer, the Block 1 block of the STDC1 module is downsampled by s tr ide = 2, and the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage 3 layer are kept unchanged to output a feature map F with a downsampling ratio of 1 / 8. Stage3 .

[0065] Step S54: transform the feature map F Stage3 Perform an average pooling to obtain the pooled feature map.

[0066] Step S55, concatenate the pooled feature map with the output feature maps of the remaining blocks of the STDC1 module in the Stage3 layer to obtain the final output feature map of the STDC1 module in the Stage3 layer.

[0067] Step S56, input the final output feature map of the STDC1 module in the Stage3 layer to the first STDC2 module in the Stage3 layer, and transfer the Block1 block of the first STDC2 module in the Stage3 layer to Block n The output feature maps of the blocks are concatenated to obtain the final output feature map of the first STDC2 module in the Stage3 layer.

[0068] Step S57, input the final output feature map of the first STDC2 module in the Stage3 layer to the second STDC2 module in the Stage3 layer, and transfer Block1 of the second STDC2 module in the Stage3 layer to Block n The output feature maps of the blocks are concatenated to obtain the final output feature map of the second STDC2 module in the Stage3 layer;

[0069] Step S58, input the final output feature map of the second STDC2 module in the Stage3 layer to the third STDC2 module in the Stage3 layer, and transfer Block1 of the third STDC2 module in the Stage3 layer to Block nThe output feature maps of the blocks are concatenated to obtain the final output feature map of the Stage 3 layer;

[0070] Step S59: Input the final output feature map of the Stage 3 layer to the Block 1 block of the STDC1 module in the Stage 4 layer for downsampling with stride = 2, and keep the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage 4 layer unchanged, so as to output a feature map F with a downsampling ratio of 1 / 16. Stage4 ;

[0071] Step S501: transform the feature map F Stage4 The Block1 block input to the STDC1 module in the Stage5 layer is downsampled with stride = 2, and the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage5 layer are kept unchanged to output a feature map F with a downsampling ratio of 1 / 32. Stage5 .

[0072] Step S502: The feature map F Stage5 Input to the PPM module to aggregate the feature map F Stage5 The context information of different regions is obtained to obtain the feature map F with global context information. Stage5-PPM Among them, the feature map F Stage5-PPM Global contextual information with larger receptive field.

[0073] Please also read Figure 5 Specifically, step S502 includes steps S5021 to S5024:

[0074] Step S5021: Use the PPM module to convert the feature map F Stage5 Four pooling operations of different sizes, 1x1, 2x2, 3x3 and 6x6, are performed respectively to obtain feature maps of four different sizes.

[0075] Step S5022: upsample the four feature maps of different sizes using bilinear interpolation until the size is equal to that of the feature map F. Stage5 The same size.

[0076] Step S5023, combine the four feature maps and the feature map F in the channel dimension Stage5 The splicing is performed to obtain a spliced ​​feature map.

[0077] Step S5024: Perform a 1x1 ConvX on the concatenated feature map to obtain a feature map F with global context information. Stage5-PPM .

[0078] Step S503: The feature map F Stage4And the feature map F Stage5 Input into the ARM module for refinement to obtain feature maps F Stage4-ARM and feature map F Stage5-ARM .

[0079] Please also read Figure 6 Specifically, step S503 includes steps S50031 to S5034:

[0080] Step S5031: transform the feature map F Stage4 Perform global pooling, BN, and sigmoid operations in sequence to generate the first attention vector;

[0081] Step S5032: The feature map F Stage4 Multiply with the first attention vector to obtain the feature map F Stage4-ARM ;

[0082] Step S5033: transform the feature map F Stage5 Perform global pooling, BN, and sigmoid operations in sequence to generate the second attention vector;

[0083] Step S5034: transform the feature map F Stage5 are multiplied by the second attention vector to obtain the feature map F Stage5-AR .

[0084] Step S504: transform the feature map F Stage5-PPM , feature map F Stage4-ARM and feature map F Stage5-ARM Input into the FFM module for feature fusion to obtain the feature map F FFM .

[0085] Please also read Figure 7 Specifically, step S504 includes steps S5041 to S5045:

[0086] Step S5041: transform the feature map F Stage5-PPM , feature map F Stage4-ARM and feature map F Stage5-ARM Perform splicing in the channel dimension to obtain a spliced ​​feature map;

[0087] Step S5042, performing a 1×1 convX operation on the concatenated feature map to obtain a feature map F1;

[0088] Step S5043, performing global pooling, 1×1 convolution, BN, and sigmoid operations on the feature map F1 in sequence to obtain a feature vector F2;

[0089] Step S5044, multiplying the feature map F1 and the feature vector F2 to obtain a feature map F3;

[0090] Step S5045, add the feature map F1 and the feature map F3 to obtain the feature map F FFM .

[0091] Step S505: transform the feature map F FFM The final segmentation result is obtained after 8-fold bilinear interpolation upsampling.

[0092] By executing steps S51 to S505, a primary image segmentation model with image segmentation and image processing functions is obtained, which is conducive to further improving the reliability and accuracy of powder spreading defect detection.

[0093] Please also read Figure 8 In some embodiments, step S90 includes steps S91 to S95.

[0094] Step S91, input the labeled test set into the final image segmentation network model to obtain the mask matrix corresponding to the image in the labeled test set. The mask matrix is ​​a three-dimensional matrix of width×height×number of channels, where width×height is the pixel coordinates corresponding to the width and height of the test image, and each channel number of each pixel coordinate corresponds to a prediction score of a defect type.

[0095] Step S92 , counting the prediction score of each pixel coordinate position in the mask matrix channel by channel, and determining the final defect type of the pixel coordinate position according to the number of channels with the highest prediction score.

[0096] Step S93, determining whether the highest prediction score is greater than a preset threshold value, if it is greater than the preset threshold value, it indicates that a defect of the final defect type exists.

[0097] Step S94, counting the number of pixels in each area of ​​the entire image where the pixel value is not 0.

[0098] In step S95, if the number of pixels is greater than a preset minimum number of pixels, it indicates that the area is a defective area.

[0099] By executing steps S91 to S95, the defect area, defect type and other information of each image in the labeled test set can be accurately obtained.

[0100] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A powder spreading defect detection method based on deep learning image segmentation, characterized in that: Includes steps: Collect the powder spreading image data after the current powder spreading; Performing defect annotation on the powder spreading image data to obtain annotated image data; Dividing the labeled image data into a labeled training set, a labeled verification set, and a labeled test set; Performing data augmentation on the labeled training set to obtain an expanded training set; Build a primary image segmentation network model; Using the labeled training set and the expanded training set as input data to perform model training on the image segmentation network model to obtain a training image segmentation network model; Inputting the labeled verification set into the training image segmentation network model, evaluating the training effect of the training image segmentation network model, and obtaining an evaluation result; the evaluation result includes a loss function value, an average intersection-over-union ratio, and an average pixel accuracy rate; The network model whose evaluation result meets the preset evaluation conditions is used as the final image segmentation network model; Inputting the labeled test set into the final image segmentation network model to output a powder spreading defect result; Among them, the preset evaluation condition is at least one of the following conditions: the loss function value gradually decreases and tends to be stable with the increase of the number of training rounds; the average intersection-over-union ratio and the average pixel accuracy gradually increase with the increase of the number of training rounds, and after the labeled verification set is input, the improvement of the average intersection-over-union ratio and the average pixel accuracy is not obvious.

2. The powder spreading defect detection method based on deep learning image segmentation according to claim 1 is characterized in that: After the step of inputting the labeled test set into the final image segmentation network model to output the powdering defect result, the step further includes: Performing image processing on the powder spreading defect results to calculate the length, width and area of ​​each defect; Determine whether the defect is within the preset control range based on the length, width and area of ​​the defect.

3. The method for powder spreading defect detection based on deep learning image segmentation according to claim 1, characterized in that: The steps of marking defects on the powder spreading image data to obtain marked image data are as follows: The different defect type regions in the powder spreading image data are annotated using an annotation tool to obtain the annotated image data.

4. The method for detecting powder spreading defects based on deep learning image segmentation according to claim 1, characterized in that: The step of performing data enhancement on the labeled training set to obtain an expanded training set includes: Establish a set of image processing methods; Using an image processing method set to perform image processing on the labeled training set for a preset number of times to obtain the expanded training set; Among them, the processing categories of the image processing method set include image geometric transformation, image color transformation, image smoothing and image noise addition; the processing methods of the image geometric transformation include horizontal flipping, vertical flipping, rotation, translation, cropping, scaling, and affine transformation; the processing methods of the image color transformation include brightness, contrast, RGB translation, hue saturation value, color equalization, hue separation, and random grid shuffling; the processing methods of the image smoothing include median filtering, Gaussian filtering, motion blur, and random smoothing; the processing methods of the image noise addition include Gaussian noise, salt and pepper noise, camera sensor noise, gamma noise, and fog addition; When establishing the image processing method set, the parameter t value is set to indicate that the image processing method of the tth category is selected to perform image processing, 0<t≤4; the parameter p1 is set to indicate the probability value of the four categories of image processing methods being randomly selected for execution; the parameter p2 is set to indicate the probability value of each processing method in a certain category of image processing methods being randomly selected for execution.

5. The method for powder spreading defect detection based on deep learning image segmentation according to claim 1, characterized in that: The primary image segmentation network model includes an image encoding module and an image decoding module; The image encoding module includes an input layer, a Stage 1 layer, a Stage 2 layer, a Stage 3 layer, a Stage 4 layer and a Stage 5 layer; The Stage1 layer and the Stage2 layer each include a ConvX module with a convolution kernel size of 3×3, and the ConvX module includes a convolution layer, a BN layer, and a ReLU activation layer; The Stage3 layer, the Stage4 layer and the Stage5 layer each include 1 STDC1 module and 3 STDC2 modules; the STDC1 module includes n Block blocks (n≥2); the STDC2 module includes n Block blocks (n≥2); The image decoding module includes a PPM module, an ARM module and a FFM module.

6. The method for powder spreading defect detection based on deep learning image segmentation according to claim 5 is characterized in that: In the STDC1 module, assuming that the number of output channels of the STDC1 module is N, the number of channels of the output feature maps of the n blocks is N / 2. i , (i=1,2,3,...,n-1), and the number of channels of the output feature map of the nth Block is the same as the number of channels of the output feature map of the n-1th Block; In the STDC2 module, assuming that the number of output channels of the STDC2 module is N, the number of channels of the output feature maps of the n blocks is N / 2. i , (i=1,2,3,...,n-1), and the number of channels of the output feature map of the nth Block is the same as the number of channels of the output feature map of the n-1th Block; The steps to build a primary image segmentation network model include: The input image is downsampled once with stride=2 at the Stage 1 layer to output a feature map F with a downsampling ratio of 1 / 2 Stage1 ; The feature map F Stage1 Perform a downsampling of s tride = 2 at the Stage 2 layer to output a feature map F with a downsampling ratio of 1 / 4 Stage2 ; The feature map F Stage2 In the Stage 3 layer, the Block 1 block of the STDC1 module is downsampled by s tr ide = 2, and the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage 3 layer are kept unchanged to output a feature map F with a downsampling ratio of 1 / 8. Stage3 ; The feature map F Stage3 Perform an average pooling to obtain the pooled feature map; The pooled feature map is concatenated with the output feature maps of the remaining blocks of the STDC1 module in the Stage3 layer to obtain the final output feature map of the STDC1 module in the Stage3 layer; Input the final output feature map of the STDC1 module in the Stage3 layer to the first STDC2 module in the Stage3 layer, and connect Block1 of the first STDC2 module in the Stage3 layer to Block n The output feature maps of the blocks are concatenated to obtain the final output feature map of the first STDC2 module in the Stage3 layer; Input the final output feature map of the first STDC2 module in the Stage3 layer to the second STDC2 module in the Stage3 layer, and connect Block1 of the second STDC2 module in the Stage3 layer to Block n The output feature maps of the blocks are concatenated to obtain the final output feature map of the second STDC2 module in the Stage3 layer; Input the final output feature map of the second STDC2 module in the Stage3 layer to the third STDC2 module in the Stage3 layer, and connect Block1 of the third STDC2 module in the Stage3 layer to Block n The output feature maps of the blocks are concatenated to obtain the final output feature map of the Stage 3 layer; The final output feature map of the Stage 3 layer is input to the Block 1 block of the STDC1 module in the Stage 4 layer for downsampling with stride = 2, and the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage 4 layer are kept unchanged to output a feature map F with a downsampling ratio of 1 / 16. Stage4 ; The feature map F Stage4 The Block1 block input to the STDC1 module in the Stage5 layer is downsampled by s tride = 2, and the width and height of the output feature map of the remaining Block blocks in the STDC1 module of the Stage5 layer are kept unchanged to output a feature map F with a downsampling ratio of 1 / 32. Stage5 ; The feature map F Stage5 Input to the PPM module to aggregate the feature map F Stage5 The context information of different regions is obtained to obtain the feature map F with global context information. Stage5-PPM ; The feature map F Stage4 And the feature map F Stage5 Input into the ARM module for refinement to obtain feature maps F Stage4-ARM and feature map F Stage5-ARM ; The feature map F Stage5-PPM , feature map F Stage4-ARM and feature map F Stage5-ARM Input into the FFM module for feature fusion to obtain the feature map F FFM ; The feature map F FFM The final segmentation result is obtained after 8-fold bilinear interpolation upsampling.

7. The method for powder spreading defect detection based on deep learning image segmentation according to claim 6, characterized in that: The feature map F Stage5 Input to the PPM module to aggregate the feature map F Stage5 The context information of different regions is obtained to obtain the feature map F with global context information. Stage5-PPM The steps include: The feature map F is transformed into Stage5 Four pooling operations of different sizes, 1x1, 2x2, 3x3 and 6x6, are performed to obtain feature maps of four different sizes; The four feature maps of different sizes are upsampled by bilinear interpolation until the size is the same as the feature map F. Stage5 The same size; The four feature maps and the feature map F are combined in the channel dimension Stage5 Perform splicing to obtain a spliced ​​feature map; Perform a 1x1 ConvX on the concatenated feature map to obtain a feature map F with global context information Stage5-PPM .

8. The method for powder spreading defect detection based on deep learning image segmentation according to claim 1, characterized in that: The feature map F Stage4 And the feature map F Stage5 Input into the ARM module for refinement to obtain feature maps F Stage4-ARM and feature map F Stage5-ARM The steps include: The feature map F Stage4 Perform global pooling, BN, and sigmoid operations in sequence to generate the first attention vector; The feature map F Stage4 Multiply it with the first attention vector to obtain the feature map F Stage4-ARM ; The feature map F Stage5 Perform global pooling, BN, and sigmoid operations in sequence to generate the second attention vector; The feature map F Stage5 are multiplied by the second attention vector to obtain the feature map F Stage5-AR .

9. The method for powder spreading defect detection based on deep learning image segmentation according to claim 1, characterized in that: The feature map F Stage5-PPM , feature map F Stage4-ARM and feature map F Stage5-ARM The steps of inputting into the FFM module for feature fusion include: The feature map F Stage5-PPM , feature map F Stage4-ARM and feature map F Stage5-ARM Perform splicing in the channel dimension to obtain a spliced ​​feature map; Perform a 1×1 convX operation on the concatenated feature map to obtain the feature map F1; Perform global pooling, 1×1 convolution, BN, and sigmoid operations on the feature map F1 in sequence to obtain the feature vector F2; Multiply the feature map F1 and the feature vector F2 to obtain the feature map F3; Add feature map F1 and feature map F3 to get feature map F FFM .

10. The method for powder spreading defect detection based on deep learning image segmentation according to claim 1, characterized in that: The step of inputting the labeled test set into the final image segmentation network model to output a powder spreading defect result comprises: Input the labeled test set into the final image segmentation network model to obtain a mask matrix corresponding to the image in the labeled test set; the mask matrix is ​​a three-dimensional matrix of width×height×number of channels, where width×height are pixel coordinates corresponding to the width and height of the test image, and each number of channels of each pixel coordinate corresponds to a prediction score of a defect type; Counting the prediction score of each pixel coordinate position in the mask matrix channel by channel, and determining the final defect type of the pixel coordinate position according to the number of channels with the highest prediction score; Determine whether the highest prediction score is greater than a preset threshold. If it is greater than the preset threshold, it indicates that a defect of the final defect type exists; Count the number of pixels in each area of ​​the entire image where the pixel value is not 0; If the number of pixels is greater than the preset minimum number of pixels, it means that the area is a defective area.

Images