Mobile-unet++ method for laser-induced damage segmentation of large-aperture optical elements

Abstract

The application discloses a Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method, and belongs to the field of optical element damage detection. The application aims at the problems of full supervision semantic segmentation of optical element laser-induced damage in the existing deep learning field, which depends on a large amount of manual annotation and has low time efficiency. The application comprises the following steps: acquiring an original damage image dataset; screening a pixel-level segmentation mask of laser-induced damage as a true value label of the original damage image, and forming a sample image set; classifying the sample images and performing data enhancement to balance the number of sample images of each category; preparing an FODI damage dataset, and determining a training set and a test set; building a Mobile-UNet++ model, and performing training and testing to obtain a trained Mobile-UNet++ model; acquiring a to-be-detected image, inputting the to-be-detected image into the trained Mobile-UNet++ model, and obtaining a semantic segmentation result. The application is used for laser-induced damage segmentation of a large-aperture optical element image in dark-field imaging.

Description

TECHNICAL FIELD

[0001] The present application relates to a Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method, and belongs to the field of optical element damage detection. BACKGROUND

[0002] Large-aperture optical elements in large laser devices are prone to laser-induced damage under high-energy laser irradiation. Rapid online detection of the damage state of the optical element is crucial for the safe and efficient operation of the ICF device. When the optical element is imaged online using dark-field side illumination technology, the damage image appears as bright spots on a dark background. The damage state of the optical element is evaluated by segmenting these bright spots online. Due to the short detection time, complex damage morphology, uneven illumination, and interference of stray light, the segmentation of the damage image is challenging.

[0003] Currently, the fully supervised semantic segmentation algorithm in the field of deep learning has achieved the most advanced segmentation performance, but it relies on a large number of high-quality pixel-level labels, which is time-consuming and labor-intensive and requires professional knowledge support. At the same time, the fully supervised semantic segmentation model with excellent performance has a large number of parameters, and a large amount of calculation results in slow data processing speed, which cannot meet the demand of detecting all optical path damage within 2 hours, and has high requirements for hardware and memory of the equipment. In 2023, the META team built a large-scale self-supervised segmentation model: Segment Anything Model (SAM), which can assist humans in quickly obtaining target segmentation masks, but it does not have semantic information.

[0004] Due to the complex scene of the optical element image and the difficulty in distinguishing the damage target, it is difficult and time-consuming to manually annotate a large number of high-quality damage labels. Moreover, the fully supervised semantic segmentation model has a large number of parameters and low time efficiency, which cannot complete the segmentation within a limited detection time. Currently, there is a lack of methods for online, fast, and accurate segmentation of laser-induced damage. SUMMARY

[0005] In view of the problem that the existing deep learning field relies on a large number of manual annotations for optical element laser-induced damage fully supervised semantic segmentation and has low time efficiency, the present application provides a Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method.

[0006] The Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method of the present application comprises,

[0007] Step 1: Obtain the original damage image dataset;

[0008] Step two: obtain a pixel-level segmentation mask of all targets in each original damage image by using SAM; screen the pixel-level segmentation mask of laser-induced damage as the true value label of the original damage image, and form a sample image set;

[0009] Step three: divide the sample images into multiple categories according to the target damage degree and the intensity and form of stray light interference, and then balance the number of sample images in each category through data enhancement;

[0010] Step four: randomly select the same number of sample images from each category to make a FODI damage data set, and divide the FODI damage data set into a training set and a test set at a ratio of 3:1;

[0011] Step five: take the backbone network of the UNet++ model as a basic architecture to build a Mobile-UNet++ model, replace the ordinary convolution in the backbone network with a depth separable convolution in the Mobile-UNet++ model, and introduce a hole convolution in the channel-by-channel convolution of the depth separable convolution; train the Mobile-UNet++ model by using the training set to obtain a trained Mobile-UNet++ model; and evaluate the segmentation performance of the trained Mobile-UNet++ model by using the test set;

[0012] Step six: obtain a to-be-detected image, input the to-be-detected image into the trained Mobile-UNet++ model, and obtain a semantic segmentation result.

[0013] According to the Mobile-UNet++ laser-induced damage segmentation method for large-aperture optical elements, the Mobile-UNet++ model comprises an encoder; the encoder comprises five feature extraction layers;

[0014] Each feature extraction layer extracts features of input features through a depth separable convolution, and the multi-channel encoding results output by the five feature extraction layers are sequentially represented as X 0,0 , X 1,0 , X 2,0 , X 3,0 and X 4,0 ;

[0015] The depth separable convolution of the first feature extraction layer extracts features of three-channel input features of a sample image to obtain the multi-channel encoding result X 0,0 output by the first feature extraction layer; the multi-channel encoding result output by the previous feature extraction layer is sequentially subjected to a maximum pooling operation, and the down-sampling features are input into the depth separable convolution of the adjacent next feature extraction layer to obtain the multi-channel encoding results X 1,0 , X 2,0 , X 3,0 and X 4,0 ;

[0016] Each depth separable convolution sequentially performs adaptive padding, per-channel atrous convolution, batch normalization operation, non-linear activation operation and multiple pointwise 1x1 convolution to expand the number of channels on the input features to obtain a multi-channel encoding result.

[0017] According to the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method, the number of pointwise 1x1 convolution channels corresponding to the five feature extraction layers is 32, 64, 128, 256 and 512 in sequence.

[0018] According to the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method, the Mobile-UNet++ model further comprises a decoder, and the decoder comprises four feature fusion layers.

[0019] Each feature fusion layer performs feature fusion through deconvolution, dense skip connection and depth separable convolution.

[0020] According to the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method, the feature fusion results corresponding to the four feature fusion layers are X 0,1 , X 0,2 , X 0,3 and X 0,4 in sequence.

[0021] The result after the deconvolution operation of X 1,0 and the fusion features after the skip connection of X 0,0 are subjected to depth separable convolution to obtain X 0,1 .

[0022] The result after the deconvolution operation of X 2,0 and the fusion features after the skip connection of X 1,0 are subjected to depth separable convolution to obtain X 1,1 ; the result after the deconvolution operation of X 1,1 and the fusion features after the skip connection of X 0,1 and X 0,0 are subjected to depth separable convolution to obtain X 0,2 .

[0023] The result after the deconvolution operation of X 3,0 and the fusion features after the skip connection of X 2,0 are subjected to depth separable convolution to obtain X 2,1 ; the result after the deconvolution operation of X 2,1 and the fusion features after the skip connection of X 1,1 and X 1,0 are subjected to depth separable convolution to obtain X 1,2 ;1,2 The result after the deconvolution operation is X 0,2 , X 0,1 and X 0,0 The fused features after the skip connection are subjected to depth separable convolution to obtain X 0,3 ;

[0024] The result after the deconvolution operation is X 4,0 The result after the deconvolution operation is X 3,0 The fused features after the skip connection are subjected to depth separable convolution to obtain X 3,1 ; X 3,1 The result after the deconvolution operation is X 2,1 and X 2,0 The fused features after the skip connection are subjected to depth separable convolution to obtain X 2,2 ; X 2,2 The result after the deconvolution operation is X 1,2 , X 1,1 and X 1,0 The fused features after the skip connection are subjected to depth separable convolution to obtain X 1,3 ; X 1,3 The result after the deconvolution operation is X 0,3 , X 0,2 , X 0,1 and X 0,0 The fused features after the skip connection are subjected to depth separable convolution to obtain X 0,4 ;

[0025] X 0,4 , X 0,3 , X 0,2 and X 0,1 are used to calculate the corresponding loss functions in combination with the true value labels during the model training process; the average error of the four loss functions is used to update the training parameters of the model through back propagation;

[0026] wherein X 0,4 is the semantic segmentation result of the current sample image.

[0027] According to the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method of the application, the method for obtaining the original damage image or the image to be detected is as follows:

[0028] The damage image of the large-aperture optical element is collected by using the terminal optical element damage detection system, and the original damage image or the image to be detected is obtained through the sliding window with the size of 128*128 and the step of 64.

[0029] In step three of the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to the present application, the target damage degree includes large-size damage and weak damage; and the stray light interference intensity and form includes strong hardware reflection, strong diffraction ring, weak hardware reflection and weak diffraction ring.

[0030] The sample images are divided into 15 categories: 1, large-size damage without stray light interference; 2, only weak damage; 3, large-size damage and strong hardware reflection; 4, large-size damage and weak hardware reflection; 5, weak damage and strong hardware reflection; 6, weak damage and weak hardware reflection; 7, large-size damage and strong diffraction ring; 8, large-size damage and weak diffraction ring; 9, weak damage and strong diffraction ring; 10, weak damage and weak diffraction ring; 11, dark background; 12, only strong hardware reflection; 13, only strong diffraction ring; 14, only weak hardware reflection; and 15, only weak diffraction ring.

[0031] In step three of the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to the present application, the data enhancement method is as follows:

[0032] The large-size damage category sample images of categories 1, 3, 4, 7 and 8 are subjected to data enhancement, and the large-size damage category sample images are superimposed on the stray light interference category images to generate new sample images.

[0033] In the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to the present application, the initial learning rate of the Mobile-UNet++ model is set to 0.001, and a decay strategy is executed; the number of samples selected each time is 1; and the number of iteration training times is 2000.

[0034] In the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to the present application, in the training process, each sample image is input into the Mobile-UNet++ model, horizontal and vertical flipping operations are performed with a probability of 50%, and an output result is obtained; in each iteration training, a cross-entropy loss function is calculated according to the output result and a true value label, a derivative of the cross-entropy loss function with respect to network parameters is calculated by back propagation, and network weights are updated in combination with an Adam optimizer.

[0035] The present application has the following advantages: the method can be used for semantic segmentation of a to-be-detected image under complex conditions such as short detection time, complex damage morphology, uneven illumination and stray light interference.

[0036] The method automatically generates an accurate pixel-level damage target mask by using a SAM large model, and assists manual production of an accurate FODI damage data label, thereby greatly reducing the label cost for training a full-supervised semantic segmentation model.

[0037] The method of the present application improves the problem of sample imbalance by adding artificially created samples for data set enhancement, and improves the ability of the network to extract effective features to distinguish stray light interference from damage points. Experimental verification shows that after data set enhancement, the recall rate of the model is improved by 27.4%, the precision rate is improved by 16.4%, and the IOU is improved by 24.1%.

[0038] In the improved lightweight Mobile-UNet++ model, the depth separable convolution is used to replace the ordinary convolution in the feature extraction process, which greatly reduces the parameter quantity of the model. At the same time, the multi-scale dilated convolution is introduced, which can expand the receptive field of the model and improve the segmentation accuracy of the model. Through verification, the parameter quantity of the method of the present application is reduced by 95.3%, 94.8% and 89.7% compared with the baseline methods: Deeplab v3, PSPNet and VGG16-Unet respectively. The segmentation accuracy of the present application is improved by 4.3% and 41.2% compared with Deeplab v3 and PSPNet respectively, and the segmentation performance is equivalent to VGG16-Unet. The Mobile-UNet++ model realizes the lightweight of the model under the premise of maintaining high segmentation accuracy, and overcomes the problem of occupying more memory and long data processing time caused by large model parameter quantity. It can run quickly and accurately on all ICF experimental platforms (devices with low computer configuration).

[0039] For large-aperture optical element damage segmentation tasks, the method of the present application fills a technical gap in the field. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flowchart of the Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method described in the present application;

[0041] Figure 2 is a network structure diagram of the Mobile-UNet++ model; in the figure represents the loss function, and ∑ is the summation symbol, representing the sum of the four loss functions; m=0, 1, 2, 3, 4; n=0, 1, 2, 3, 4;

[0042] Figure 3 is a data processing flowchart of the depth separable convolution; N is the number of point-by-point convolutions, i.e. the number of output feature channels;

[0043] Figure 4 is a schematic diagram of the pixel-level true value label of the sample image (FODI image);

[0044] Figure 5 is a schematic diagram of the target and stray light interference in the sample image;

[0045] Figure 6A schematic diagram of 15 classes of sample images;

[0046] Figure 7 A schematic diagram of the FODI damage data set;

[0047] Figure 8 A schematic diagram of the segmentation results of the method of the present application and the baseline method;

[0048] Figure 9 A schematic diagram of the segmentation results of the laser-induced damage of the full-aperture optical element. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0050] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0051] The present application will be further described below in combination with the drawings and specific embodiments, but is not limited to the present application.

[0052] DETAILED DESCRIPTION Figures 1 to 7 As shown in the drawings, the present application provides a Mobile-UNet++ laser-induced damage segmentation method for large-aperture optical elements, comprising,

[0053] Step 1: Obtain the original damage image data set;

[0054] Step 2: Obtain the pixel-level segmentation mask of all targets in each original damage image using SAM; manually screen the pixel-level segmentation mask of the laser-induced damage as the true value label of the original damage image, and form a sample image set;

[0055] Step 3: According to the target damage degree and the intensity and form of stray light interference, the sample images are divided into multiple categories, and then through data enhancement, the number of sample images in each category is balanced;

[0056] Step 4: Randomly select the same number of sample images from each category to make a FODI damage data set, and divide it into a training set and a test set in a ratio of 3:1;

[0057] Step five: a Mobile-UNet++ model is built based on the backbone network of the UNet++ model, in the Mobile-UNet++ model, the ordinary convolution in the backbone network is replaced by a depth separable convolution, and a hole convolution is introduced in the channel-by-channel convolution of the depth separable convolution, so as to effectively improve the receptive field of the model, so that the output of the convolution layer can cover more image features than ordinary convolution; at the same time, more convolution numbers are added in the point-by-point convolution (1*1 convolution) in the depth separable convolution, so as to expand the channel number of the features.

[0058] The Mobile-UNet++ model is trained by using the training set, and a trained Mobile-UNet++ model is obtained; the segmentation performance of the trained Mobile-UNet++ model is evaluated by using the test set.

[0059] Step six: obtaining a to-be-detected image, inputting the to-be-detected image into the trained Mobile-UNet++ model, and obtaining a semantic segmentation result.

[0060] Further, as shown in Figure 2 and Figure 3 , the Mobile-UNet++ model comprises an encoder; the encoder comprises five feature extraction layers;

[0061] Each feature extraction layer extracts features from input features through a depth separable convolution, and the multi-channel encoding results output by the five feature extraction layers are sequentially represented as X 0,0 , X 1,0 , X 2,0 , X 3,0 and X 4,0 .

[0062] The depth separable convolution of the first feature extraction layer extracts features from three-channel input features of a sample image, and obtains a multi-channel encoding result X 0,0 output by the first feature extraction layer; the multi-channel encoding result output by the previous feature extraction layer is sequentially subjected to a maximum pooling operation, and the down-sampling features are input into the depth separable convolution of the adjacent next feature extraction layer, so as to obtain multi-channel encoding results X 1,0 , X 2,0 , X 3,0 and X 4,0 .

[0063] In the embodiment, the specific obtaining methods of X 1,0 , X 2,0 , X 3,0 and X 4,0 are as follows:

[0064] The multi-channel encoding result X 0,0The down-sampling feature obtained by the max-pooling operation is input into a deep separable convolution of a second feature extraction layer for feature extraction, to obtain a multi-channel encoding result X output by the second feature extraction layer 1,0 The down-sampling feature obtained by the max-pooling operation is input into a deep separable convolution of a third feature extraction layer for feature extraction, to obtain a multi-channel encoding result X output by the third feature extraction layer 1,0 2,0 2 ,0 The down-sampling feature obtained by the max-pooling operation is input into a deep separable convolution of a fourth feature extraction layer for feature extraction, to obtain a multi-channel encoding result X output by the fourth feature extraction layer 3,0 3,0 The down-sampling feature obtained by the max-pooling operation is input into a deep separable convolution of a fifth feature extraction layer for feature extraction, to obtain a multi-channel encoding result X output by the fifth feature extraction layer 4,0 .

[0065] Each deep separable convolution sequentially performs adaptive padding, channel-wise atrous convolution, BatchNorm2d operation, ReLU operation, and multiple point-wise 1x1 convolution for expanding the number of channels on the input feature to obtain a multi-channel output feature, and then performs max-pooling operation to obtain a multi-channel encoding result.

[0066] As an example, the number of channels of the point-wise 1x1 convolution corresponding to the five feature extraction layers is 32, 64, 128, 256, and 512, respectively.

[0067] Further, as shown in Figure 2 and Figure 3 , the Mobile-UNet++ model further includes a decoder, and the decoder includes four feature fusion layers.

[0068] Each feature fusion layer performs feature fusion through deconvolution, dense skip connection, and deep separable convolution.

[0069] The specific process of feature fusion is as follows:

[0070] The feature fusion results corresponding to the four feature fusion layers are X 0,1 , X 0,2 , X 0,3 , and X 0,4 , respectively.

[0071] The result after deconvolution operation on X 1,0 is fused with the fused feature after skip connection on X 0,0 to obtain X 0,1 through deep separable convolution. ​​​

[0072] X 2,0 the result after performing the deconvolution operation is X 1,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 1,1 ; X 1,1 the result after performing the deconvolution operation is X 0,1 and X 0,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 0,2 ;

[0073] X 3,0 the result after performing the deconvolution operation is X 2,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 2,1 ; X 2,1 the result after performing the deconvolution operation is X 1,1 and X 1,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 1,2 ; X 1,2 the result after performing the deconvolution operation is X 0,2 , X 0,1 and X 0,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 0,3 ;

[0074] X 4,0 the result after performing the deconvolution operation is X 3,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 3,1 ; X 3,1 the result after performing the deconvolution operation is X 2,1 and X 2,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 2,2 ; X 2,2 the result after performing the deconvolution operation is X 1,2 , X 1,1 and X 1,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 1,3 ; X 1,3 the result after performing the deconvolution operation is X 0,3 , X 0,2 , X 0,1 and X 0,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 0,4 ;

[0075] X 0,4 , X 0,3 , X 0,2 and X 0,1The loss functions are used to calculate the corresponding loss functions in combination with the ground truth labels during model training; the average error of the four loss functions is used for backpropagation to update the training parameters of the model.

[0076] Where X 0,4 This represents the semantic segmentation result of the current sample image.

[0077] As an example, the method for obtaining the original damage image or the image to be detected is as follows:

[0078] Damage images of large-aperture optical elements are acquired using a front-end optical element damage detection system (FODI). The original damage image or the image to be detected is obtained by cropping through a sliding window with a size of 128×128 and a step size of 64.

[0079] Furthermore, combining Figure 5 As shown, in step three, the degree of target damage includes large-size damage and weak damage; the intensity and form of stray light interference include strong hardware reflection, strong diffraction ring, weak hardware reflection and weak diffraction ring.

[0080] The sample images were divided into 15 categories: 1. Large-sized damage without stray light interference; 2. Only minor damage; 3. Large-sized damage and strong hardware reflection; 4. Large-sized damage and weak hardware reflection; 5. Weak damage and strong hardware reflection; 6. Weak damage and weak hardware reflection; 7. Large-sized damage and strong diffraction rings; 8. Large-sized damage and weak diffraction rings; 9. Weak damage and strong diffraction rings; 10. Weak damage and weak diffraction rings; 11. Dark background; 12. Only strong hardware reflection; 13. Only strong diffraction rings; 14. Only weak hardware reflection; 15. Only weak diffraction rings. Categories 1-10 are damage categories, and categories 11-15 are background categories.

[0081] As an example, the final sample image set (FODI damage dataset) may include 659 training images, of which 100 are artificially augmented. The test set contains 218 images across 15 classes.

[0082] In step three of this embodiment, the method for data augmentation is as follows:

[0083] If the number of sample images in a certain category is significantly less than that in other categories, data augmentation is needed to increase the number of samples and balance the number of sample images in different categories. It has been verified that the number of large-size damage sample images in categories 1, 3, 4, 7 and 8 is relatively small, so image augmentation needs to be performed separately for each category. The specific method is to overlay the large-size damage sample images with low occurrence rate onto the stray light interference images to generate new sample images.

[0084] As an example, for the training of the Mobile-UNet++ model, the initial learning rate is set to 0.001, and a decay strategy is performed; the number of samples selected each time is 1; the number of iterations of training is 2000.

[0085] During the training process of the present embodiment, each sample image is input into the Mobile-UNet++ model, and a horizontal and vertical flipping operation is performed with a probability of 50% to obtain an output result; in each iteration of training, the cross-entropy loss function (BCE) is calculated according to the output result and the true value label, the derivative of the cross-entropy loss function with respect to the network parameters is calculated by back propagation, and the network weights are updated in combination with the Adam (Adaptive Moment Estimation) optimizer.

[0086] After training a group of data to update the model weight, the method for evaluating the segmentation performance of the trained Mobile-UNet++ model using the test set includes: inputting the test set data into the Mobile-UNet++ model to obtain a semantic segmentation result; and calculating segmentation indicators including segmentation precision (Precision), segmentation recall (Recall), segmentation F1 (F1), false detection rate (FDR) and segmentation intersection over union (IoU) through the true value label.

[0087] The training is ended after saving the training model parameters with the best segmentation indicators.

[0088] During actual detection, the damage image of a large-aperture optical element is collected online by a terminal optical element damage detection system (FODI) and is subjected to cropping processing. The image to be detected is input into the trained Mobile-UNet++ model with loaded model parameters to obtain a semantic segmentation result; the damage target region is segmented by thresholding; and all regions are spliced to obtain a full-aperture FODI damage image segmentation result.

[0089] The calculation method of the segmentation indicators is as follows:

[0090] Segmentation precision:

[0091]

[0092] In the formula, TP is a true positive, that is, the number of pixels segmented as damage by the model and actually damaged; FP is a false positive, that is, the number of pixels segmented as damage by the model but not actually damaged;

[0093] Segmentation recall:

[0094]

[0095] In the formula, FN is a false negative, that is, the number of pixels segmented as non-damage by the model but actually damaged;

[0096] Segmentation F1 value:

[0097]

[0098] Segmentation IoU:

[0099]

[0100] In the formula, groud-truth pixs is the number of pixels of the true value label; predicted pixels is the number of pixels segmented as damage by the model.

[0101] In combination Figure 8 and Figure 9 , the Mobile-UNet++ and the baseline method segmentation evaluation index comparison results are obtained, as shown in Table 1:

[0102] Table 1

[0103]

[0104] In the table, Methods represents the method, and Params represents the parameter amount.

[0105] Then, the Mobile-UNet++ and the non-lightweight UNet++ evaluation index are compared, and Table 2 is obtained:

[0106]

[0107] Although the present application is described herein with reference to particular embodiments, it is to be understood that these examples are merely illustrative of the principles and applications of the present application. It is therefore to be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the present application as defined by the appended claims. It is to be understood that different dependent claims and features described herein can be combined with each other in ways other than those described in the original claims. It is also to be understood that features described in relation to one individual embodiment can be used in other described embodiments.

Claims

1. A Mobile-UNet++ laser-induced damage segmentation method for large-aperture optical elements, characterized by The application relates to a Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method. Step one: obtaining an original damage image dataset; Step two: obtaining a pixel-level segmentation mask of all targets in each original damage image by using SAM; screening a pixel-level segmentation mask of laser-induced damage as a true value label of the original damage image, and forming a sample image set; Step three: according to target damage degree and stray light interference intensity and form, the sample images are divided into multiple categories, and then data enhancement is performed to balance the number of sample images in each category; Step four: a same number of sample images are randomly selected from each category to form a FODI damage dataset, and the FODI damage dataset is divided into a training set and a test set at a ratio of 3:1; Step five: a Mobile-UNet++ model is built based on a backbone network of a UNet++ model, ordinary convolution in the backbone network is replaced with depth separable convolution, and a hole convolution is introduced into a channel-by-channel convolution of the depth separable convolution; The Mobile-UNet++ model is trained by using the training set to obtain a trained Mobile-UNet++ model, and the segmentation performance of the trained Mobile-UNet++ model is evaluated by using the test set; Step six: obtaining a to-be-detected image and inputting the to-be-detected image into the trained Mobile-UNet++ model to obtain a semantic segmentation result; The Mobile-UNet++ model comprises an encoder; the encoder comprises five feature extraction layers; Each feature extraction layer extracts features of the input features through a depth separable convolution, and multi-channel encoding results output by the five feature extraction layers are sequentially represented as X 0,0 , X 1,0 , X 2,0 , X 3,0 , and X 4,0 ; The depth separable convolution of the first feature extraction layer extracts features of three channel input features of the sample image to obtain a multi-channel encoding result X output by the first feature extraction layer 0,0 ; the multi-channel encoding result output by the previous feature extraction layer is sequentially subjected to a max-pooling operation, and the down-sampling features are input to a depth separable convolution of an adjacent next feature extraction layer to obtain multi-channel encoding results X 1,0 , X 2,0 , X 3,0 , and X 4,0 ; Each depth separable convolution sequentially performs adaptive padding, channel-by-channel hole convolution, batch normalization, nonlinear activation and multiple point-by-point 1*1 convolution to expand the number of channels and obtain a multi-channel encoding result.

2. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 1, characterized in that, The number of channels of the point-by-point 1*1 convolution corresponding to the five feature extraction layers is 32, 64, 128, 256 and 512 in sequence.

3. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 2, characterized in that, The Mobile-UNet++ model further comprises a decoder, and the decoder comprises four feature fusion layers; Each feature fusion layer performs feature fusion through deconvolution, dense skip connection and depth separable convolution.

4. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 3, characterized in that, The feature fusion results corresponding to the four feature fusion layers are X 0,1 , X 0,2 , X 0,3 , and X 0,4 ; X 1,0 The result after performing the deconvolution operation on X 0,0 The fused features after performing the skip connection are subjected to a depth separable convolution to obtain X 0,1 ; X 2,0 the result after performing the deconvolution operation is X 1,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 1,1 X 1,1 the result after performing the deconvolution operation is X 0,1 and X 0,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 0,2 ; X 3,0 the result after the deconvolution operation is X 2,0 the fused feature after the skip connection is subjected to a depth separable convolution to obtain X 2,1 ; X 2,1 the result after the deconvolution operation is X 1,1 and X 1,0 the fused feature after the skip connection is subjected to a depth separable convolution to obtain X 1,2 ; X 1,2 the result after the deconvolution operation is X 0,2 , X 0,1 and X 0,0 the fused feature after the skip connection is subjected to a depth separable convolution to obtain X 0,3 ; X 4,0 the result after performing the deconvolution operation and X 3,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 3,1 ; X 3,1 the result after performing the deconvolution operation and X 2,1 and X 2,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 2,2 ; X 2,2 the result after performing the deconvolution operation and X 1,2 , X 1,1 and X 1,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 1,3 ; X 1,3 the result after performing the deconvolution operation and X 0,3 , X 0,2 , X 0,1 and X 0,0 the fused feature after performing the skip connection is subjected to a depth separable convolution to obtain X 0,4 ; X 0,4 , X 0,3 , X 0,2 and X 0,1 for calculating the corresponding loss function respectively in combination with the true value label during the model training process; the average error of the four loss functions is used for updating the training parameters of the model in reverse propagation; where X 0,4 is the semantic segmentation result of the current sample image.

5. The Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to claim 4, wherein The method for obtaining the original damage image or the to-be-detected image is as follows: An optical element damage detection system is used to collect damage images of a large-aperture optical element, and a sliding window with a size of 128*128 and a step of 64 is used for cropping to obtain the original damage image or the to-be-detected image.

6. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 5, characterized in that, In step three, the target damage degree includes large-size damage and weak damage, and the stray light interference intensity and form includes strong hardware reflection, strong diffraction ring, weak hardware reflection and weak diffraction ring; The sample images are divided into 15 categories:

1. large-size damage without stray light interference; 2. only weak damage; 3. large-size damage and strong hardware reflection; 4. large-size damage and weak hardware reflection; 5. weak damage and strong hardware reflection; 6. weak damage and weak hardware reflection; 7. large-size damage and strong diffraction ring; 8. large-size damage and weak diffraction ring; 9. weak damage and strong diffraction ring; 10, weak damage and weak diffraction ring; 11, dark background; 12, only strong hardware reflection; 13, only strong diffraction ring; 14, only weak hardware reflection; 15, only weak diffraction ring.

7. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 6, characterized in that, In step three, the method for data enhancement is: The large-size damage class sample images of categories 1, 3, 4, 7 and 8 are subjected to data enhancement, and the large-size damage class sample images are superimposed on the stray light interference class images to generate new sample images.

8. The Mobile-UNet++ large aperture optical element laser-induced damage segmentation method according to claim 7, characterized in that, In the training of the Mobile-UNet++ model, the initial learning rate is set to 0.001, and the decay strategy is executed; the number of samples selected each time is 1; and the number of iteration training is 2000.

9. The Mobile-UNet++ large-aperture optical element laser-induced damage segmentation method according to claim 8, characterized in that, During the training process, each sample image is input into the Mobile-UNet++ model, horizontal and vertical flipping operations are performed with a probability of 50%, and an output result is obtained; In each iteration training, a cross-entropy loss function is calculated according to the output result and the true value label, the derivative of the cross-entropy loss function with respect to the network parameters is calculated by back propagation, and the network weights are updated in combination with the Adam optimizer.

Images