A deep learning-based carbon filament surface defect detection method

Abstract

The application discloses a kind of carbon filament surface defect detection methods based on deep learning, including convolution layer is extracted to original image carbon filament feature, filter background information, generate multiple feature maps;Multiple feature maps are sent into encoder module and are encoded;After feature map coding, high and low layer features of carbon filament defect are extracted and fused using backbone network, generate multiple scale feature maps;Multiple scale feature maps generated by backbone network are sent into decoder module and are decoded;The feature map generated after decoding of decoder is fused with the result feature map after preliminary convolution, for the classification and regression prediction of carbon filament defect;The results of classification prediction and regression prediction are output.The application reduces the interference of human subjectivity, which greatly contributes to improving the production speed of large carbon filament bundle, ensuring the quality of carbon filament products and reducing the work intensity of labor personnel.

Description

Technical Field

[0001] This invention belongs to the field of carbon filament surface defect detection technology, specifically relating to a carbon filament surface defect detection method based on deep learning. Background Technology

[0002] Key research and development and application of new materials include the key research and development of carbon fiber and its composite materials, breakthroughs in preparation, evaluation, and application of core technologies. Against the backdrop of rapid industrialization worldwide, the applications of carbon fiber are becoming increasingly diversified, with surging demand in aerospace, key new materials research and application, infrastructure construction, and large buildings. It is a crucial component of aero-engine blades, launch vehicles, wind turbine blades, and concrete reinforcement materials.

[0003] During carbon fiber fabrication, defects such as fuzzy fibers, tangled fibers, and joints caused by improper manufacturing processes not only affect the strength and appearance of the carbon fiber but also significantly reduce the tensile strength, abrasion resistance, and production stability of the carbon fiber products. Surface defect detection of carbon fiber is typically performed manually, but this method is labor-intensive, inefficient, and produces inconsistent results. Deep learning, by mimicking human visual, auditory, thinking, and responsive activities, has solved many complex pattern recognition problems and achieved remarkable results in data mining, machine learning, natural language processing, neuromorphic computing, and speech recognition. Therefore, deep learning can also achieve highly accurate detection of carbon fiber defects. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a carbon filament surface defect detection method based on deep learning. This method collects carbon filament bundle surface image data and achieves efficient, accurate, and intelligent carbon filament defect identification and localization based on the collected images. This method makes a significant contribution to reducing the workload of workers and improving the intelligence and automation of industrial production.

[0005] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A deep learning-based method for detecting surface defects in carbon filaments, comprising:

[0007] Step 1: The convolutional layer extracts carbon fiber features from the original image, filters background information, and generates multiple feature maps;

[0008] Step 2: Input multiple feature maps into the encoder module for encoding;

[0009] Step 3: After encoding the feature map, the backbone network is used to extract and fuse the high and low layer features of carbon filament defects to generate multi-scale feature maps;

[0010] Step 4: The multi-scale feature maps generated by the backbone network are fed into the decoder module for decoding;

[0011] Step 5: Fuse the feature map generated by the decoder with the feature map resulting from the initial convolution for classification and regression prediction of carbon filament defects;

[0012] Step 6: Output the results of classification and regression predictions.

[0013] To optimize the above technical solution, the specific measures also include:

[0014] Step 1 above uses a multi-head structure to process the feature extraction task separately, extracting defect features from the input carbon filament raw image to obtain a feature map;

[0015] Each head consists of three convolutional layers, and the defect feature f H The extraction process is represented as follows:

[0016] f H =H i (x)

[0017] Where x is the input original carbon fiber image, H i This represents the head structure of the i-th element.

[0018] Step 2 above includes:

[0019] Step (2.1) divides the feature map into blocks f pi The system is divided into n blocks, and a learnable positional code E is added to each block. pi , where the position code E pi The location of the segment determines E. pi +f pi The input to the encoder yields:

[0020] y0 = [E p1 +f p1 E p2 +f p2 ,...,E pn +f pn ]

[0021] y0 is E pi and f pi The sum and initial vector;

[0022] Step (2.2): The initial vector is fed into the MSA self-attention module for processing to obtain y′. i :

[0023] y′ i =MSA(q i ,k i ,vi )+y i-1

[0024] q i =k i =v i =LN(y i-1 )

[0025] Where LN represents the normalization operation, q i k i v i After remembering the normalization result from the previous layer, it is fed into MSA. The result processed by MSA is then added to the encoded result y from the previous layer. i-1 Get y′ i ;

[0026] Step (2.3): y′ i After normalization, the result is fed into FFN. The result after FFN is added to the result y′ after MSA processing. i The encoded result y is obtained. i The encoding process needs to be repeated multiple times until it is complete.

[0027] y i =FFN(LN(y′) i ))+y′ i

[0028] [f E1 ,f E2 ,...,f En ]=y l

[0029] Wherein, FFN represents a feedforward neural network;

[0030] [f E1 ,f E2 ,...,f En ] represents the encoded feature;

[0031] l represents the number of layers in the encoder.

[0032] Step 3 above uses the Feature Pyramid Network (FPN) based on the ResNet architecture as the backbone network.

[0033] The decoder described in step 4 above consists of two multi-head self-attention modules (MSA) and a feedforward neural network (FFN). Step 4 uses the decoder to learn decoding features for different tasks, and the specific process is as follows:

[0034] Step (4.1): Perform the first self-attention module processing to obtain z′ i ;

[0035] z0 = [f E1 ,f E2 ,...,f En ]

[0036] q i =k i =v i =LN(z i-1 )

[0037] z′ i =MSA(q i ,k i ,v i )+z i-1

[0038] Where z0 represents the input of the decoder, which is the feature map output generated after the encoder's encoding result is passed through the backbone network;

[0039] Where LN represents the normalization operation, q i k i v i This represents the normalization result of the previous layer;

[0040] MSA stands for Multi-Head Self-Attention Module. After processing by the MSA module, the input z of the upper-layer decoder is added. i-1 Get z′ i ;

[0041] Step (4.2): Perform a second self-attention module process to finally obtain z″. i :

[0042] z″ i =MSA(q′) i ,k′ i ,v′ i )+z′ i

[0043] q′ i ,k′ i ,v′ i The calculation formula is as follows:

[0044] q′ i =LN(z′) i ), k′ i =v′ i =LN(z0)

[0045] Where LN represents the normalization operation, MSA represents the multi-head self-attention module, and the result after the second MSA processing is added to the output z′ after the first self-attention module processing. i Finally, we get z″ i ;

[0046] Step (4.3): Move z″ i After normalization, the result is fed into the feedforward neural network FFN, and the result is then added to the result of two multi-head self-attention modules MSA, z″. i The final decoding result z is obtained. i ;

[0047] z i =FFN(LN(z″) i ))+z″ i

[0048] [f D1 ,f D2 ,...,f Dn ] = z l

[0049] Where l represents the number of layers in the decoder, f Di This is the output of the decoder.

[0050] In step 5 above, the optimal match between the predicted bounding box and the ground truth bounding box is generated using a loss function, which is designed as follows:

[0051] L=λ cls ·L cls +λ L1 ·L L1 +λ giou ·L giou

[0052] Among them, L cls For predicting the classification and ground truth class labels, the Focal Loss function, L L1 and L giou Let λ be the normalized L1 loss function and GIOU loss function for the center coordinates, length, and width of the predicted bounding box and the ground truth bounding box, respectively. cls , λ L1 , λ giou These are the coefficients of each loss function.

[0053] The present invention has the following beneficial effects:

[0054] This invention encodes the learned carbon fiber defect features through a multi-layer encoder; decodes the learned defect features through a multi-layer decoder; and trains the encoder-decoder-based two-stage method EBD-CNN on a carbon fiber dataset to achieve carbon fiber defect recognition and localization.

[0055] 1. This invention uses an encoder-decoder based convolutional neural network EBD-CNN to complete the task of detecting minute defects in large areas of carbon fiber bundles, which improves the accuracy, speed and intelligence of defect detection. It is a fast, efficient and intelligent processing method.

[0056] 2. Adding an encoder-decoder process during feature transmission can minimize information loss during feature transmission and improve the detection accuracy of carbon wire surface defects.

[0057] 3. This method achieves high-precision detection of minute carbon filament defects in large carbon filament bundles while reducing the interference of human subjectivity. This makes a significant contribution to improving the production speed of large carbon filament bundles, ensuring the quality of carbon filament products, and reducing the workload of workers. Attached Figure Description

[0058] Figure 1 This is a diagram of the EBD-CNN network structure of the present invention. Detailed Implementation

[0059] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0060] See Figure 1 A deep learning-based method for detecting surface defects in carbon filaments includes:

[0061] Step 1: The convolutional layer extracts carbon fiber features from the original image, filters background information, and generates multiple feature maps;

[0062] A multi-head structure is used to process the feature extraction task separately, extracting defect features from the input raw carbon filament image to obtain a feature map;

[0063] Each head consists of three convolutional layers, and the defect feature f H The extraction process is represented as follows:

[0064] f H =H i (x)

[0065] Where x is the input original carbon fiber image, H i This represents the head structure of the i-th element (containing three convolution processes).

[0066] Step 2: Input multiple feature maps into the encoder module for encoding;

[0067] Step (2.1) divides the feature map into blocks f pi The system is divided into n blocks, and a learnable positional code E is added to each block. pi , where the position code E pi The location of the segment determines E. pi +f pi The input to the encoder yields:

[0068] y0 = [E p1 +f p1E p2 +f p2 ,...,E pn +f pn ]

[0069] y0 is E pi and f pi The sum and initial vector;

[0070] Step (2.2): The initial vector is fed into the MSA (Multi-head self-attention) self-attention module for processing to obtain y′. i :

[0071] y′ i =MSA(q i ,k i ,v i )+y i-1

[0072] q i =k i =v i =LN(y i-1 )

[0073] Where LN represents the normalization operation, q i k i v i After remembering the normalization result of the previous layer (i-1 layer), it is fed into MSA. The result processed by MSA is then added to the encoded result y of the previous layer. i-1 Get y′ i ;

[0074] Step (2.3): y′ after processing by the multi-head self-attention module MSA i After normalization, the result is fed into FFN. The result after FFN is added to the result y′ after MSA processing. i To obtain the encoding result y of a certain layer i The encoding process needs to be repeated multiple times until it is complete.

[0075] y i =FFN(LN(y′) i ))+y′ i

[0076] [f E1 ,f E2 ,...,f En ]=y l

[0077] Wherein, FFN represents a feedforward neural network;

[0078] [f E1,f E2 ,...,f En ] represents the encoded feature;

[0079] l represents the number of layers in the encoder.

[0080] Step 3: After encoding the feature map, a Feature Pyramid Network (FPN) based on the ResNet architecture is used as the backbone network to extract and fuse the high and low layer features of carbon filament defects to generate multi-scale feature maps.

[0081] Step 4: The multi-scale feature maps generated by the backbone network are fed into the decoder module for decoding;

[0082] The decoder consists of two multi-head self-attention modules (MSA) and a feedforward neural network (FFN).

[0083] Step 4 involves learning the decoding features for different tasks using a decoder. The specific process is as follows:

[0084] Step (4.1): Perform the first self-attention module processing to obtain z′ i ;

[0085] z0 = [f E1 ,f E2 ,...,f En ]

[0086] q i =k i =v i =LN(z i-1 )

[0087] z′ i =MSA(q i ,k i ,v i )+z i-1

[0088] Where z0 represents the input of the decoder, which is the feature map output generated after the encoder's encoding result is passed through the backbone network;

[0089] Where LN represents the normalization operation, q i k i v i This represents the normalization result of the previous layer (i-1 layers);

[0090] Similarly, q i k i v i The input z is fed into the MSA (Multi-Head Self-Attention Module), which processes the data and adds it to the input z of the upper-layer decoder. i-1 Get z′i ;

[0091] Step (4.2): Perform a second self-attention module process to finally obtain z″. i :

[0092] z″ i =MSA(q′) i ,k′ i ,v′ i )+z′ i

[0093] q′ i ,k′ i ,v′ i The calculation formula is as follows:

[0094] q′ i =LN(z′) i ), k′ i =v′ i =LN(z0)

[0095] Where LN represents the normalization operation, MSA represents the multi-head self-attention module, and the result after the second MSA processing is added to the output z′ after the first self-attention module processing. i Finally, we get z″ i ;

[0096] Step (4.3): Move z″ i After normalization, the result is fed into the feedforward neural network FFN, and the result is then added to the result of two multi-head self-attention modules MSA, z″. i The final decoding result z is obtained. i ;

[0097] z i =FFN(LN(z″) i ))+z″ i

[0098] [f D1 ,f D2 ,...,f Dn ] = z l

[0099] The decoding process is performed the same number of times as the encoding process, where l represents the number of layers in the decoder, and f Di This is the output of the decoder. The codec minimizes information loss during feature transmission.

[0100] Step 5: Fuse the feature map generated by the decoder with the feature map resulting from the initial convolution for classification and regression prediction of carbon filament defects;

[0101] The loss function can be used to generate the optimal match between the predicted bounding box and the ground truth bounding box. The loss function is designed as follows:

[0102] L=λ cls ·L cls +λ L1 ·L L1 +λ giou ·L giou

[0103] Among them, L cls For predicting the classification and ground truth class labels, the Focal Loss function, L L1 and L giou Let λ be the normalized L1 loss function and GIOU loss function for the center coordinates, length, and width of the predicted bounding box and the ground truth bounding box, respectively. cls , λ L1 , λ giou These are the coefficients of each loss function.

[0104] Step 6: Output the results of classification and regression predictions.

[0105] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in carbon filaments based on deep learning, characterized in that, include: Step 1: The convolutional layer extracts carbon fiber features from the original image, filters background information, and generates multiple feature maps; Step 2: Input multiple feature maps into the encoder module for encoding, including: Step (2.1) divides the feature map into blocks. It is divided into Each block is assigned a learnable positional code. Among them, position encoding Determined by the location of the segmented blocks, The input to the encoder yields: yes and The sum and initial vector; Step (2.2): Feed the initial vector into the multi-head self-attention module Processing is performed to obtain : in This indicates a normalization operation. , , After remembering the normalization result from the previous layer, it is fed into the multi-head self-attention module. Through multi-head self-attention module The processed result plus the encoding result from the previous layer get ; Step (2.3): After normalization, the data is fed into the feedforward neural network. After passing through a feedforward neural network The results after the operation, plus the multi-head self-attention module The result after processing The encoded result is obtained. The encoding process needs to be repeated multiple times until it is complete. in, This represents a feedforward neural network; Represents the encoded features; Indicates the number of layers in the encoder; Step 3: After encoding the feature map, the backbone network is used to extract and fuse the high and low layer features of carbon filament defects to generate multi-scale feature maps; Step 4: The multi-scale feature maps generated by the backbone network are fed into the decoder module for decoding; the decoder consists of two multi-head self-attention modules. and a feedforward neural network The process involves learning decoding features for different tasks using a decoder, as detailed below: Step (4.1): Perform the first multi-head self-attention module processing to obtain... ; in, This represents the input to the decoder, which is the feature map output generated after the encoder's encoding result is passed through the backbone network. in, This indicates a normalization operation. , , This represents the normalization result of the previous layer; This represents a multi-head self-attention module, after which... After processing, the input of the upper-layer decoder is added. get ; Step (4.2): Perform a second multi-head self-attention module processing to finally obtain... : The calculation formula is as follows: in, This indicates a normalization operation. This indicates a multi-head self-attention module, followed by a second multi-head self-attention module. The processed result plus the output after the first multi-head self-attention module processing Finally obtained ; Step (4.3): After normalization, it is then fed into the feedforward neural network. The results, plus two multi-head self-attention modules, Processing results To obtain the final decoding result ; in, Indicates the number of layers in the decoder. This is the output of the decoder; Step 5: Fuse the feature map generated by the decoder with the feature map resulting from the initial convolution for classification and regression prediction of carbon filament defects; Step 6: Output the results of classification and regression predictions.

2. The method for detecting surface defects of carbon filaments based on deep learning according to claim 1, characterized in that, Step 1 employs a multi-head structure to process the feature extraction task separately, extracting defect features from the input raw carbon fiber image to obtain a feature map; Each head consists of three convolutional layers, defect features The extraction process is represented as follows: in, The input is the original carbon fiber image. This represents the head structure of the i-th element.

3. The method for detecting surface defects of carbon filaments based on deep learning according to claim 1, characterized in that, Step 3 uses the Feature Pyramid Network (FPN) based on the ResNet architecture as the backbone network.

4. The method for detecting surface defects of carbon filaments based on deep learning according to claim 1, characterized in that, In step 5, the optimal match between the predicted bounding box and the ground truth bounding box is generated using a loss function, which is designed as follows: in, Focal Loss function is used to predict the classification and ground truth class labels. and These are the normalized L1 loss function and GIOU loss function for the center coordinates, length, and width of the predicted bounding box and the ground truth bounding box, respectively. , , These are the coefficients of each loss function.

Images