Semi-supervised optical fiber defect detection method combining multipath disturbance branch algorithm and uncertainty perception network

Abstract

The invention relates to a semi-supervised optical fiber defect detection method combining a multipath disturbance branch algorithm and an uncertainty perception network, which comprises the following steps: preprocessing optical fiber image data to obtain a training data set; wherein the training data set comprises labeled image data and unlabeled image data; inputting the training data set into a semi-supervised learning model, and training the semi-supervised learning model to obtain an optical fiber defect detection model; wherein the semi-supervised learning model comprises a multipath disturbance branch algorithm module and an uncertainty sensing network module; and performing optical fiber defect detection by using the optical fiber defect detection model. According to the method, the defect identification accuracy in the optical fiber manufacturing process can be improved, the yield of optical fiber production is greatly improved, and an important solution is provided for improving the quality of optical fiber products and controlling the production cost.

Description

Technical Field

[0001] The present invention relates to the technical field of quality inspection in optical fiber production, and particularly to a semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty-aware network. Background Art

[0002] Optical fibers support the operation of modern communication networks and play an irreplaceable role in the global informatization process. However, in large-scale manufacturing production scenarios, due to factors such as production processes, raw materials, external environments, and machinery and equipment, it is inevitable that the produced optical fibers will have certain defects. Optical fiber defect detection means are the key to restricting optical fiber quality inspection, which can effectively reduce production costs, improve product quality and production efficiency. On optical fibers, different defects indicate different reasons for the occurrence of defects, and by analyzing the type of defects, the problems that occur in the production process can be traced.

[0003] Traditional optical fiber defect detection is carried out by professional personnel through visual inspection or customized professional instruments. In recent years, deep learning technology has been increasingly widely used in the field of optical fiber defect detection due to its powerful feature extraction ability and efficient processing of complex background images. The semi-supervised object detection network model is a typical representative of deep learning. It only requires a small amount of labeled data, effectively utilizes limited labeled data and a large amount of unlabeled information for model training, and realizes defect detection. In the optical fiber defect task, the input of the semi-supervised object detection network model is the optical fiber defect data set, and the output end will give the classification and location of the defects, which has greatly improved the efficiency compared with traditional methods. Summary of the Invention

[0004] The purpose of the present invention is to propose a semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty-aware network for optical fiber defect detection, so as to improve the accuracy and recall rate of optical fiber defect detection.

[0005] To achieve the above purpose, the present invention provides the following solutions:

[0006] The present invention proposes a semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty-aware network, including:

[0007] Preprocess the optical fiber image data to obtain a training data set; wherein, the training data set includes: labeled image data and unlabeled image data;

[0008] Input the training data set into a semi-supervised learning model, train the semi-supervised learning model, and obtain an optical fiber defect detection model; wherein, the semi-supervised learning model includes: a multi-path perturbation branch algorithm module and an uncertainty-aware network module;

[0009] Use the optical fiber defect detection model to perform optical fiber defect detection.

[0010] Optionally, the multi-branch perturbation algorithm module includes: a multi-branch teacher model and a student model. The multi-branch teacher model and the student model are object detection networks with the same structure. Among them, the weights of the teacher model are obtained by calculating the exponential moving average of the weights of the student model. The object detection network includes: a backbone network CSPDarknet, a feature pyramid network PAFPN network in the detection head part, and a Head network.

[0011] Optionally, the multi-branch includes: a weak image enhancement branch for labeled data, a strong image enhancement branch for labeled data, a weak image enhancement branch for unlabeled data, and a strong image enhancement branch for unlabeled data; among them, weak image enhancement only includes image enhancement based on geometric transformation, and strong image enhancement includes image enhancement based on geometric transformation and image enhancement based on color transformation;

[0012] Only the weak image enhancement branch for unlabeled data is output by the teacher model, and the other three branches are all processed by the student model; and, the output of the weak image enhancement branch for unlabeled data is processed by non-maximum suppression with a confidence threshold to generate pseudo-labels for unlabeled data, which are used to assist the learning of unlabeled data.

[0013] Optionally, the uncertainty-aware network module includes: a foreground and background feature uncertainty module and a bounding box uncertainty module; among them, the foreground is the optical fiber defect, and the background is the area outside the optical fiber defect;

[0014] The foreground and background feature uncertainty module includes: a foreground and background attention information extractor FB, a CBR network, and an auxiliary training head. The CBR network includes a convolutional layer, a batch normalization layer, and a ReLU activation layer;

[0015] The foreground and background attention information extractor FB is used to multiply the input feature map with the foreground and background attention feature maps respectively to obtain a foreground-enhanced feature map and a background-enhanced feature map, then subtract the background-enhanced feature map from the input feature map, and fuse the foreground-enhanced feature map to obtain the finally output feature map;

[0016] The bounding box uncertainty module includes: a fully connected layer and a random jitter function;

[0017] The bounding box uncertainty module is used to obtain the loss function box L -Loss and box J -Loss, which measures the localization quality of the model for the predicted target.

[0018] Optionally, inputting the training data set into the semi-supervised learning model includes:

[0019] Performing image augmentation with different strengths on the labeled image data to obtain the image data of the weak image augmentation branch of the labeled data and the strong image augmentation branch of the labeled data;

[0020] Inputting the image data of the weak image augmentation branch of the labeled data and the strong image augmentation branch of the labeled data into the student model and the foreground and background feature uncertainty module to obtain the defect classification of the optical fibers in the two branches of the labeled data, the confidence of the optical fiber defects, and the localization boxes of the optical fiber defects;

[0021] Performing image augmentation with different strengths on the unlabeled image data to obtain the image data of the weak image augmentation branch of the unlabeled data and the strong image augmentation branch of the unlabeled data;

[0022] Randomly fusing the data of the weak image augmentation branch of the labeled data in the image data of the strong image augmentation branch of the unlabeled data to obtain the fused image data;

[0023] Inputting the fused image data into the student model and the foreground and background attention information extractor FB for target classification and localization prediction;

[0024] Inputting the image data of the weak image augmentation branch of the unlabeled data into the teacher model and the foreground and background attention information extractor FB to obtain the defect classification, defect confidence, and defect localization box of the unlabeled image; using the output of the weak image augmentation branch of the unlabeled data as pseudo-labels, and fusing the labels corresponding to the weak image augmentation branch images of the labeled data in the same situation of randomly fusing the weak image augmentation branch images of the labeled data and the strong image augmentation branch images of the unlabeled data to supervise and assist the training of the strong image augmentation branch of the unlabeled data.

[0025] Optionally, inputting the image data of the weak image augmentation branch of the labeled data and the strong image augmentation branch of the labeled data into the student model and the foreground and background feature uncertainty module includes:

[0026] Inputting the image data of the weak image augmentation branch of the labeled data and the strong image augmentation branch of the labeled data into the backbone network of the student model respectively for feature extraction, and transmitting the extracted multi-scale feature maps to the foreground and background attention information extractor FB;

[0027] The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature maps and transmits them to the PAFPN network. The PAFPN network then fuses the feature maps, and the fused feature maps are transmitted to the Head network. The Head network finally performs object classification and localization prediction, and the output is the defect classification of the optical fiber, the confidence of the optical fiber defect, and the localization box of the optical fiber defect.

[0028] The optical fiber defect localization box output by the strong image enhancement branch of the labeled data is processed by the MLP network and the random jitter function to obtain two loss functions for the uncertainty of the localization box, so as to optimize the defect localization ability of the model. At the same time, the loss function of the strong image enhancement branch of the labeled data and the loss function of the weak image enhancement branch of the labeled data are regularized to improve the robustness of the model.

[0029] Optionally, inputting the fused image data into the student model and the foreground and background attention information extractor FB includes:

[0030] Input the fused image data into the backbone network of the student model for feature extraction, and transmit the extracted multi-scale feature maps to the foreground and background attention information extractor FB;

[0031] The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature maps and transmits them to the PAFPN network. The PAFPN network then fuses the feature maps, and the fused feature maps are transmitted to the Head network. The Head network finally performs object classification and localization prediction.

[0032] Optionally, inputting the image data of the weak image enhancement branch of the unlabeled data into the teacher model and the foreground and background attention information extractor FB includes:

[0033] Input the fused image data into the backbone network of the teacher model for feature extraction, and transmit the extracted multi-scale feature maps to the foreground and background attention information extractor FB;

[0034] The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature maps and transmits them to the PAFPN network. The PAFPN network then fuses the feature maps, and the fused feature maps are transmitted to the Head network. The Head network finally performs object classification and localization prediction.

[0035] Optionally, inputting the training data set into the semi-supervised learning model further includes:

[0036] Input the labeled image data into the backbone network of the student model for feature extraction. After the extracted feature maps are processed by two different CBR networks, a foreground attention feature map and a background attention feature map are obtained;

[0037] The foreground attention feature map and the background attention feature map are output through the auxiliary training head to obtain the predicted foreground mask and background mask. The predicted foreground mask, background mask, the labeled foreground mask, and background mask are calculated through the Dice-Loss function to obtain the foreground mask loss fg-Loss and the background mask loss bg-Loss; The feature map output by the backbone network of the student model is subtracted from the background attention feature map to obtain a weakened background feature map, and the weakened background feature map is compared and calculated with the foreground enhanced feature map to form a contrast loss function Con-Loss.

[0038] The beneficial effects of the present invention are:

[0039] In a semi-supervised fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty-aware network, the present invention proposes a novel semi-supervised learning model that can make full use of a small amount of labeled data and a large amount of unlabeled data to detect fiber defects. The multi-path perturbation branch algorithm realizes consistency regularization constraints, effectively utilizes the information of unlabeled data, and can better extract the defect features of the fiber; The uncertainty-aware network can fully alleviate the problems of inter-class similarity and intra-class dissimilarity in fiber defect detection, and can effectively improve the defect feature extraction ability and defect localization ability, thereby more accurately detecting fiber defects. The present invention can improve the recognition accuracy of defects in the fiber manufacturing process, greatly improve the yield of fiber production, and provide an important solution for improving the quality of fiber products and controlling production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a schematic diagram of fiber defect detection according to an embodiment of the present invention;

[0042] Figure 2 It is a semi-supervised learning model according to an embodiment of the present invention;

[0043] Figure 3 It is a foreground and background attention information extractor FB of the semi-supervised learning model according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] As Figure 1 shown, this embodiment proposes a semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network, including:

[0047] Preprocess the optical fiber image data to obtain a training data set; wherein, the training data set includes: labeled image data and unlabeled image data;

[0048] Input the training data set into a semi-supervised learning model, train the semi-supervised learning model to obtain an optical fiber defect detection model; wherein, the semi-supervised learning model includes: a multi-path perturbation branch algorithm module and an uncertainty perception network module;

[0049] Use the optical fiber defect detection model to perform optical fiber defect detection.

[0050] Specifically, in this embodiment, first preprocess the optical fiber data set images and divide them into a training set and a test set; secondly, input the training set into the designed defect detection model and use the training set to train the defect detection model; finally, use the test set to test the trained defect detection model, and use the Intersection Over Union (IoU), Precision, and Recall to evaluate the model performance.

[0051] In the preprocessing process of the optical fiber data set, convert the optical fiber image data into an image with a size of 640×320 and a pixel channel of 3; the ratio of the training set to the test set in the data set is 4:1, and the ratio of the labeled images to the unlabeled images in the training set is 1:3.

[0052] The semi-supervised learning model designed in this embodiment includes two parts: a multi-path perturbation branch algorithm and an uncertainty perception network;

[0053] The multi-branch perturbation algorithm consists of a multi-branch teacher-student network model. The multi-branch teacher model and the student model are object detection networks with the same structure, where the weights of the teacher model are calculated from the exponential moving average of the weights of the student model. This object detection network mainly consists of the following parts: the backbone network CSPDarknet (Cross Stage Partial Darknet), the feature pyramid network PAFPN (Path Aggregation Feature Pyramid Networks) in the detection head part, and the Head network.

[0054] During the specific object detection network processing, the image is first input into CSPDarknet for feature extraction. The multi-scale feature maps extracted by CSPDarknet are passed to PAFPN, and PAFPN then fuses the feature maps. The fused feature maps are passed to the Head network, and the Head network finally performs target classification and localization prediction.

[0055] CSPDarknet is responsible for extracting the features of the image. It consists of a convolutional layer, a batch normalization layer, a SiLU activation function, and a max pooling layer. The input dimension is 640×640×3, and the output contains three scales with dimensions of 80×80×256, 40×40×512, and 20×20×1024 respectively. PAFPN is responsible for fusing the features extracted by the backbone network. It consists of a convolutional layer, a batch normalization layer, a SiLU activation function, and an upsampling function. The input of PAFPN is the three different-scale feature maps output by CSPDarknet. PAFPN fuses the different-scale feature maps, and the output also contains three scales with dimensions of 80×80×256, 40×40×512, and 20×20×1024 respectively. The Head network is responsible for performing target classification and localization prediction using the fused feature maps. It consists of three branches: a classification branch for predicting the probability distribution of the target category, a regression branch for predicting the target position information, and an object branch for predicting the target confidence. Specifically, it consists of three convolutional layers. The input of the Head network is the three different-scale feature maps output by PAFPN. The Head network makes predictions, and the output is the defect classification of the optical fiber, the confidence of the optical fiber defect, and the localization box of the optical fiber defect.

[0056] The multi-branch teacher-student network is characterized in that the data in the training set is divided into four different branches for processing. The four different branches are: the weak image enhancement branch of labeled data, the strong image enhancement branch of labeled data, the weak image enhancement branch of unlabeled data, and the strong image enhancement branch of unlabeled data. Among them, weak image enhancement only includes image enhancement based on geometric transformation, and strong image enhancement includes image enhancement based on geometric transformation and image enhancement based on color transformation.

[0057] Among the four branches, only the weak image enhancement branch of unlabeled data is output by the teacher model, and the other three branches are processed by the student model. The weak image enhancement branch of unlabeled data is output by the teacher model. After non-maximum suppression processing with a confidence threshold, pseudo-labels for unlabeled data are generated to assist in the learning of unlabeled data.

[0058] The relationship between the four different branches is to constrain the output of the weak image enhancement branch and the strong image enhancement branch of labeled data through a regularization loss function to achieve consistency regularization; during the training process of the strong enhancement branch of unlabeled data, image-label pairs with lower loss functions in the weak image enhancement branch of labeled data are randomly fused to improve the robustness of the model.

[0059] The uncertainty-aware network is divided into two parts: the foreground and background feature uncertainty module and the bounding box uncertainty module.

[0060] The foreground and background feature uncertainty module consists of a foreground and background attention information extractor FB, a CBR network, and an auxiliary training head constructed by a convolutional layer. The CBR network includes a convolutional layer, a batch normalization layer, and a ReLU activation layer;

[0061] The foreground and background attention information extractor FB is characterized in that the input feature map is multiplied by the foreground and background attention feature maps respectively to obtain the foreground-enhanced feature map and the background-enhanced feature map. Then, the input feature map is subtracted from the background-enhanced feature map, and after fusing the foreground-enhanced feature map, the finally output feature map is obtained;

[0062] The weight generation process of the foreground and background attention information extractor is as follows: The features extracted by the backbone network CSPDarknet are processed by different CBR networks and then input into the auxiliary training head. After calculation with the foreground mask and the background mask, the foreground mask loss fg-Loss and the background mask loss bg-Loss are obtained; the features extracted by the backbone network CSPDarknet are subtracted from the background attention feature map to obtain the weakened background feature map. The weakened background feature map is compared with the foreground-enhanced feature map to obtain the foreground contrast loss function Con-Loss. The trained foreground and background attention feature maps are the weights F fg and F bg .

[0063] The bounding box uncertainty module consists of a fully connected layer and a random jitter function.

[0064] The bounding box uncertainty module is characterized in that the bounding box output of the strong image enhancement branch of labeled data is processed by a fully connected layer and a random jitter function to obtain the loss function box for bounding box uncertaintyL -Loss and box J -Loss.

[0065] Furthermore, inputting the training data set into the semi-supervised learning model includes:

[0066] Performing image enhancement with different strengths on the labeled image data to obtain the image data of the weak image enhancement branch of the labeled data and the strong image enhancement branch of the labeled data;

[0067] Inputting the image data of the weak image enhancement branch of the labeled data and the strong image enhancement branch of the labeled data into the student model and the foreground and background feature uncertainty module to obtain the defect classification of the optical fibers in the two branches of the labeled data, the confidence of the optical fiber defects, and the localization box of the optical fiber defects;

[0068] Performing image enhancement with different strengths on the unlabeled image data to obtain the image data of the weak image enhancement branch of the unlabeled data and the strong image enhancement branch of the unlabeled data;

[0069] Randomly fusing the data of the weak image enhancement branch of the labeled data in the image data of the strong image enhancement branch of the unlabeled data to obtain the fused image data;

[0070] Inputting the fused image data into the student model and the foreground and background attention information extractor FB for object classification and localization prediction;

[0071] Inputting the image data of the weak image enhancement branch of the unlabeled data into the teacher model and the foreground and background attention information extractor FB to obtain the defect classification, defect confidence, and defect localization box of the unlabeled image; taking the output of the weak image enhancement branch of the unlabeled data as the pseudo-label, and fusing the label corresponding to the weak image enhancement branch image of the labeled data in the same situation of randomly fusing the weak image enhancement branch image of the labeled data and the strong image enhancement branch image of the unlabeled data to supervise and assist the training of the strong image enhancement branch of the unlabeled data.

[0072] More specifically, the semi-supervised learning model proposed in this embodiment is as Figure 2 shown; the semi-supervised learning model is composed of a student model, a teacher model, a foreground and background attention information extractor FB network, a CBR network, an MLP network, and an auxiliary training head. Among them, the student model and the teacher model are composed of object detection networks with the same structure and different weight parameters. The weight of the teacher model is calculated from the exponential moving average of the weight of the student model. The object detection network is mainly composed of the following parts: the backbone network CSPDarknet, the PAFPN and Head networks in the detection head part.

[0073] Specific semi-supervised learning model processing process:

[0074] (1) The labeled images are input into the backbone network CSPDarknet of the student model for feature extraction. The multi-scale feature maps extracted by CSPDarknet are passed to the CBR network, and foreground and background attention feature maps are obtained through training with the auxiliary training head;

[0075] (2) The labeled image data undergoes image enhancement with different strengths to obtain the image data of the weak image enhancement branch and the strong image enhancement branch of the labeled data. The image data of these two branches are input into the backbone network CSPDarknet of the student model for feature extraction. The multi-scale feature maps extracted by CSPDarknet are passed to the foreground and background attention information extractor FB. FB enhances the foreground features of the feature maps and passes them to PAFPN. PAFPN then fuses the feature maps, and the fused feature maps are passed to the Head network. The Head network finally performs target classification and localization prediction, and the output is the defect classification of the optical fiber, the confidence of the optical fiber defect, and the localization box of the optical fiber defect;

[0076] (3) The optical fiber defect localization box output by the strong image enhancement branch of the labeled data is processed by the MLP network (Multilayer Perceptron) and the random jitter function to obtain two loss functions regarding the uncertainty of the localization box, so as to optimize the model's defect localization ability. At the same time, the loss functions of the strong image enhancement branch of the labeled data and the weak image enhancement branch of the labeled data are subjected to regularization constraints to improve the robustness of the model. Moreover, the exponential moving average of the weight parameters of the student model is transferred to the teacher model;

[0077] (4) The unlabeled image data undergoes image enhancement with different strengths, obtaining the image data of the weak image enhancement branch of the unlabeled data and the strong image enhancement branch of the unlabeled data. The data of the strong image enhancement branch of the unlabeled data is randomly fused with the data of the weak image enhancement branch of the labeled data to improve the model robustness. The former is input into the backbone network CSPDarknet of the teacher model, and the latter is input into the backbone network CSPDarknet of the student model. The multi-scale feature maps extracted by CSPDarknet are passed to the foreground and background attention information extractor FB, which enhances the foreground features of the feature maps and passes them to PAFPN. PAFPN then fuses the feature maps, and the fused feature maps are passed to the Head network. Finally, the Head network performs target classification and localization prediction. After the output of the Head network of the teacher model undergoes non-maximum suppression processing with confidence, the defect classification, defect confidence, and defect localization box of the unlabeled image are obtained. This output serves as the pseudo-label, which fuses the label corresponding to the weak image enhancement branch image of the labeled data in the same random fusion situation of the weak image enhancement branch image of the labeled data and the strong image enhancement branch image of the unlabeled data to supervise and assist the training of the strong image enhancement branch of the unlabeled data.

[0078] S207 is the backbone network of the student model, whose structure is the backbone network CSPDarknet, and the inputs are the images of the weak image enhancement branch of the labeled data, the images of the strong image enhancement branch of the labeled data, and the images of the strong image enhancement branch of the unlabeled data. The input fiber optic dataset needs to be preprocessed and extracted into images with a size of 640×320 and 3 channels. The output of S207 is processed by the foreground and background attention information extractor FB of S209 and then input into the S210 student model detection head. The S210 student model detection head is composed of PAFPN and the Head network.

[0079] I s f Is the output of the strong image enhancement branch of the labeled data, which consists of three parts: fiber optic defect classification, fiber optic defect confidence, and fiber optic defect localization box. I s f Is obtained after the strongly enhanced image of the labeled data is processed by the backbone network S207 of the student model, the foreground and background attention information extractor FB of S209, and the S210 student model detection head.

[0080] S213 is for I s f The defect localization box of, after being processed by the S214 MLP network, obtains the S219 loss function box L -Loss, after being processed by the S215 random jitter function, obtains the S220 loss function boxJ -Loss, loss function box L -Loss and box J -Loss constitutes the loss function about the uncertainty of the positioning box. S221 is I s f The total loss function I s f -Loss, including fiber defect classification, fiber defect confidence, and fiber defect positioning frame, consists of three parts. The fiber defect classification and fiber defect confidence parts are calculated by Focal-Loss, and the fiber defect positioning frame part is calculated by CIoU-Loss.

[0081] I s It is the weak image enhancement branch output with labeled data, which consists of three parts: fiber defect classification, fiber defect confidence, and fiber defect positioning frame. s It is a weakly enhanced image with labeled data obtained after being processed by S207, which is the backbone network of the student model, S209, the foreground and background attention information extractor FB, and S210, the student model detection head.

[0082] S223 is 1 s The total loss function I s -Loss, including fiber defect classification, fiber defect confidence, and fiber defect location frame, a total of three parts. The fiber defect classification and fiber defect confidence parts are calculated by Focal-Loss, and the fiber defect location frame part is calculated by CIoU-Loss. S222 is a regularization constraint, which is constructed by L2 distance and aims to s f Total loss function S221 and I s The total loss function S223 is pulled closer to achieve consistency regularization.

[0083] The CBR network structure of S205 and S206 consists of a convolutional layer, a batch normalization layer, and a ReLU activation layer. S202 is an auxiliary training head constructed by a convolutional layer. The feature map output by the backbone network of the student model for labeled data passes through two different CBR networks of S205 and S206, and then the foreground attention feature map S203 and the background attention feature map S204 are obtained. S203 and S204 pass through the auxiliary training head of S202 to output the predicted foreground mask and background mask. The predicted foreground mask, background mask, and the labeled foreground mask and background mask are calculated through the Dice-Loss function to obtain the foreground mask loss fg-Loss of S217 and the background mask loss bg-Loss of S212. The feature map output by the backbone network of the student model is subtracted from S204 to obtain a weakened background feature map, and the weakened background feature map is compared with the foreground enhanced feature map to calculate and form the contrast loss function Con-Loss of S218.

[0084] S208 is the backbone network of the teacher model, and its structure is the backbone network CSPDarknet. Its weights are calculated by the exponential moving average of the weights of the backbone network of the student model in S207. The input is the image of the weak image enhancement branch of unlabeled data. The input fiber optic dataset needs to be preprocessed and extracted into an image with a size of 640×320 and 3 channels.

[0085] The output of S208 is processed by the foreground and background attention information extractor FB of S209 and then input to the detection head of the teacher model in S211. The detection head of the teacher model in S211 consists of a PAFPN and a Head network, and its weights are calculated by the exponential moving average of the weights of the detection head of the student model in S210.

[0086] S201 is a method of randomly fused image enhancement. S201 randomly fuses I s The image with a lower Loss in the labeled weak image enhancement branch and the image of the strong image enhancement branch of unlabeled data are fused to obtain a fused image to alleviate the adverse phenomenon of overfitting. The fused image passes through the backbone network of the student model in S207, the foreground and background attention information extractor FB of S209, and the detection head of the student model in S210, and then I t f , I t f is the output of the strong image enhancement branch of unlabeled data. I t is the output of the weak image enhancement branch of unlabeled data. I t After passing through the non-maximum suppression processing with a confidence threshold in S216, the pseudo-label of unlabeled data is obtained. S225 is the same random fusion process as S201, and this pseudo-label and I sRandomly fuse the labels corresponding to the images with lower Loss in the weakly labeled image enhancement to obtain fused pseudo-labels. I t f -Loss is about I t f The total loss function with the fused pseudo-labels consists of three parts: fiber defect classification, fiber defect confidence, and fiber defect localization box. The fiber defect classification and fiber defect confidence parts are calculated by Focal-Loss, and the fiber defect localization box part is calculated by CIoU-Loss.

[0087] S209 is the foreground and background attention information extractor FB, and the specific structure is as shown in the appendix Figure 3 shown. S301 is the feature map output by the backbone network of the S207 student model, and S302 and S303 are the foreground attention feature map of S203 and the background attention feature map of S204 respectively. The input feature map of S301 is multiplied by the foreground and background attention feature maps respectively to obtain the foreground and background enhanced feature maps. Then, the input feature map is subtracted from the background enhanced feature map to obtain the background weakened feature map, and the background weakened feature map and the foreground enhanced feature map are fused to obtain the final output.

[0088] The embodiments described above are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network, characterized in that: include: Preprocessing the optical fiber image data to obtain a training data set; wherein the training data set includes: labeled image data and unlabeled image data; Inputting the training data set into a semi-supervised learning model, training the semi-supervised learning model, and obtaining an optical fiber defect detection model; wherein the semi-supervised learning model includes: a multi-path perturbation branch algorithm module and an uncertainty perception network module; The optical fiber defect detection model is used to perform optical fiber defect detection.

2. According to claim 1, the semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network is characterized in that: The multi-path perturbation branch algorithm module includes: a multi-branch teacher model and a student model. The multi-branch teacher model and the student model are target detection networks with the same structure, wherein the weight of the teacher model is calculated by the exponential moving average of the student model weight. The target detection network includes: a backbone network CSPDarknet, a feature pyramid network PAFPN network of the detection head part, and a Head network.

3. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 2 is characterized in that: The multiple branches include: weak image enhancement branch with labeled data, strong image enhancement branch with labeled data, weak image enhancement branch with unlabeled data, and strong image enhancement branch with unlabeled data; among them, weak image enhancement only includes image enhancement based on geometric transformation, and strong image enhancement includes image enhancement based on geometric transformation and image enhancement based on color transformation; Only the weak image enhancement branch of unlabeled data is output by the teacher model, and the other three branches are processed by the student model. Moreover, the output of the weak image enhancement branch of unlabeled data is processed by non-maximum suppression with a confidence threshold to generate pseudo labels for the unlabeled data to assist the learning of unlabeled data.

4. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 3 is characterized in that: The uncertainty perception network includes: a foreground and background feature uncertainty module and a positioning frame uncertainty module; wherein the foreground is the fiber defect and the background is the area outside the fiber defect; The foreground and background feature uncertainty module includes: foreground and background attention information extractor FB, CBR network, auxiliary training head, and the CBR network includes convolution layer, batch normalization layer and ReLU activation layer; The foreground and background attention information extractor FB is used to multiply the input feature map with the foreground and background attention feature maps to obtain a foreground enhanced feature map and a background enhanced feature map, respectively, and then subtract the background enhanced feature map from the input feature map, and fuse the foreground enhanced feature map to obtain a final output feature map; The positioning frame uncertainty module includes: MLP network and random jitter function; The positioning box uncertainty module is used to obtain the loss function box about the uncertainty of the bounding box after the bounding box output of the strong image enhancement branch with labeled data is processed by the fully connected layer and the random jitter function. L -Loss and box J -Loss, which measures the quality of the model’s positioning of the predicted target.

5. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 4 is characterized in that: Inputting the training data set into the semi-supervised learning model includes: The labeled image data is subjected to image enhancement of different strengths to obtain image data of a weak image enhancement branch of labeled data and a strong image enhancement branch of labeled data; Input the image data of the weak image enhancement branch with labeled data and the strong image enhancement branch with labeled data into the student model and the foreground and background feature uncertainty module to obtain the defect classification of the optical fiber, the confidence of the optical fiber defect, and the positioning frame of the optical fiber defect of the two branches with labeled data; The unlabeled image data is subjected to image enhancement of different strengths to obtain image data of a weak image enhancement branch of the unlabeled data and a strong image enhancement branch of the unlabeled data; Randomly fuse the data of the weak image enhancement branch with labeled data into the image data of the strong image enhancement branch with unlabeled data to obtain fused image data; The fused image data is input into the student model and the foreground and background attention information extractor FB to perform target classification and location prediction; The image data of the weak image enhancement branch of the unlabeled data is input into the teacher model and the foreground and background attention information extractor FB to obtain the defect classification, defect confidence, and defect location box of the unlabeled image; the output of the weak image enhancement branch of the unlabeled data is used as a pseudo-label, and the pseudo-label is the same as the random fusion of the labeled weak image enhancement branch image and the unlabeled strong image enhancement branch image, and the label corresponding to the weak image enhancement branch image of the labeled data is fused to supervise and assist the training of the strong image enhancement branch of the unlabeled data.

6. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 5, characterized in that: Inputting the image data of the weak image enhancement branch with labeled data and the image data of the strong image enhancement branch with labeled data into the student model and the foreground and background feature uncertainty module includes: The image data of the weak image enhancement branch with labeled data and the image data of the strong image enhancement branch with labeled data are respectively input into the backbone network of the student model for feature extraction, and the extracted multi-scale feature map is passed to the foreground and background attention information extractor FB; The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature map and passes it to the PAFPN network. The PAFPN network then fuses the feature map, and the fused feature map is passed to the Head network. The Head network finally classifies and locates the target, and the output is the fiber defect classification, fiber defect confidence, and fiber defect positioning frame; The fiber defect localization frame output by the strong image enhancement branch with labeled data is processed by the MLP network and the random jitter function to obtain two loss functions about the uncertainty of the localization frame to optimize the defect localization ability of the model. At the same time, the loss function of the strong image enhancement branch with labeled data and the loss function of the weak image enhancement branch with labeled data are regularized to improve the robustness of the model.

7. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 5, characterized in that: The fused image data is input into the student model and the foreground and background attention information extractor FB including: The fused image data is input into the backbone network of the student model for feature extraction, and the extracted multi-scale feature map is passed to the foreground and background attention information extractor FB; The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature maps and passes them to the PAFPN network. The PAFPN network then fuses the feature maps, and the fused feature maps are passed to the Head network. The Head network finally performs target classification and positioning prediction.

8. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 5, characterized in that: The image data of the weak image enhancement branch of the unlabeled data is input into the teacher model and the foreground and background attention information extractor FB including: The fused image data is input into the backbone network of the teacher model for feature extraction, and the extracted multi-scale feature map is passed to the foreground and background attention information extractor FB; The foreground and background attention information extractor FB enhances the foreground features of the extracted multi-scale feature maps and passes them to the PAFPN network. The PAFPN network then fuses the feature maps, and the fused feature maps are passed to the Head network. The Head network finally performs target classification and positioning prediction.

9. The semi-supervised optical fiber defect detection method combining a multi-path perturbation branch algorithm and an uncertainty perception network according to claim 5, characterized in that: Inputting the training data set into the semi-supervised learning model further comprises: The labeled image data is input into the backbone network of the student model for feature extraction. The extracted feature maps are processed by two CBR networks with different weights to obtain a foreground attention feature map and a background attention feature map; The foreground attention feature map and the background attention feature map are output by the auxiliary training head to obtain the predicted foreground mask and background mask. The predicted foreground mask, background mask and the labeled foreground mask, background mask are calculated through the Dice-Loss function to obtain the foreground mask loss fg-Loss and the background mask loss bg-Loss, which respectively measure the backbone network and the CBR network's ability to extract foreground and background features; the feature map output by the backbone network of the student model is subtracted from the background attention feature map to obtain the weakened background feature map, and the weakened background feature map is compared with the foreground enhancement feature map to form the contrast loss function Con-Loss, which assists in judging the backbone network and the CBR network's ability to extract foreground and background features.

Images