Carbon fiber prepreg surface defect detection method based on CFP-SSD model

Abstract

The present invention discloses a carbon fiber prepreg surface defect detection method based on the CFP-SSD model. First, an image acquisition device is used to collect the prepreg surface defect images generated during the CFP preparation process in real time, and a carbon fiber prepreg image dataset is established. Then, a backbone feature extraction network module, an additional feature layer extraction module, and a multi-scale fusion long-distance context feature extraction module are designed to construct a CFP-SSD carbon fiber prepreg surface defect detection model. Then, the loss function and training hyperparameters of the CFP-SSD model are set. The designed CFP-SSD is trained by feeding the carbon fiber prepreg image dataset to obtain a trained CFP-SSD model. Finally, the trained CFP-SSD model is used to detect carbon fiber prepreg surface defects. The method of the present invention has strong recognition and positioning capabilities for defects of different sizes, and especially improves the recognition capability of small-scale defects.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent manufacturing and relates to a carbon fiber prepreg surface defect detection method based on a CFP-SSD model. Background Art

[0002] Carbon fiber prepreg (CFP) is a mid- and downstream product in the carbon fiber industry. Featuring high strength, low density, excellent plasticity, corrosion resistance, and a long lifespan, it has been widely used in high-end fields such as aerospace, military, and new energy. However, during production, carbon fiber prepreg is susceptible to surface defects due to various factors, including the production process.

[0003] At present, the detection of carbon fiber prepreg surface quality mainly relies on manual qualitative identification. Workers manually label and record the size, type and location of prepreg defects. The efficiency and accuracy of workers' subjective defect calibration cannot be guaranteed.

[0004] Traditional composite material defect detection methods include ultrasonic testing, laser testing, X-ray testing, and infrared testing. Ultrasonic testing requires a coupling agent as a transmission medium, is a contact method, and is slow and time-consuming, making it unsuitable for production line requirements. X-ray testing is harmful to the human body and insensitive to defects parallel to the material surface, resulting in poor detection results. These methods are unsuitable for large-scale deployment due to high equipment costs and slow detection speeds.

[0005] In recent years, machine vision inspection technology has developed rapidly. It boasts advantages such as non-contact, high speed, high precision, and strong on-site anti-interference capabilities. It has replaced a large amount of manpower, improving inspection efficiency while also completing tasks that are beyond the reach of human labor. It is widely used in industrial inspection processes and has promising application prospects. Traditional machine vision methods identify defects through steps such as image grayscale transformation, smoothing filtering, threshold segmentation, edge detection, and contour extraction. This manual feature extraction method has poor transferability and accuracy in the field of defect detection, and the workload is large, gradually failing to meet the needs of the manufacturing industry. Therefore, research on methods for rapid and intelligent location and identification of surface defects during the preparation of carbon fiber prepregs is beneficial for timely feedback on issues and process adjustments.

[0006] As a classic single-stage target detector, the SSD model offers advantages such as fast detection speed, low computational complexity, and high detection accuracy. While no prior art has found applications for surface defect detection in carbon fiber prepregs, the SSD model has been used for surface defect detection in other fields. While the SSD model uses multi-scale feature maps to effectively predict defects of varying sizes, the independence of features at different scales results in a lack of interaction between shallow detail information and deep semantic information. This is particularly evident when detecting small targets, as the extracted feature maps are incomplete and semantic information is severely lost, resulting in poor detection results and hindering feature extraction within convolutional neural networks. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems existing in the prior art and to provide a method for detecting surface defects of carbon fiber prepregs based on the CFP-SSD model. The backbone feature extraction network of the SSD model is VGG16 and the feature maps of different scales generated by the SSD model are independent of each other and have no interaction; the CFP-SSD model designed by the present invention has a backbone feature extraction network module that is the improved Resnet50. Compared with the VGG16 model, the feature extraction effect is better. At the same time, the multi-scale fusion long-distance context feature extraction module in the CFP-SSD model effectively combines feature maps of different scales, realizing effective interaction between shallow and deep information. These two improvements significantly improve the detection accuracy of the CFP-SSD model compared to the traditional SSD model, especially for the detection of small-scale defects.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A carbon fiber prepreg surface defect detection method based on the CFP-SSD model includes the following steps:

[0010] (1) Using image acquisition equipment to collect carbon fiber prepreg images in real time during the production process, manually mark the defects in the carbon fiber prepreg images with rectangular frames, generate corresponding label data for each carbon fiber prepreg image, and establish a carbon fiber prepreg image dataset;

[0011] (2) Expanding the carbon fiber prepreg image dataset, using a portion of the expanded carbon fiber prepreg image dataset as a training set, and compressing the original images;

[0012] (3) Constructing the CFP-SSD model;

[0013] The CFP-SSD model consists of a backbone feature extraction network module, an additional feature layer extraction module, and a multi-scale fusion long-distance context feature extraction module;

[0014] The backbone feature extraction network module uses an improved Resnet50 network. The specific improvement method is as follows: the channel attention mechanism module SENet layer is introduced into the Resnet50 structure to recalibrate the channel weights, and the important image features are enhanced to improve the network feature extraction effect. The channel attention mechanism module SENet layer is added after the first three conv2_x layers, conv3_x layers, and conv4_x layers of Resnet50, and finally the weighted recalibrated feature map P1 is obtained;

[0015] The prior art generally introduces the channel attention mechanism module SENet layer into the Resnet (such as Resnet50, Resnet101, etc.) trunk feature extraction network by adding a SENet module to the trunk branch of each residual layer in the Resnet structure. Compared with the prior art, the present invention introduces the channel attention mechanism module SENet layer into the Resnet50 feature extraction network. The method has the advantages of being simple and intuitive, having a good feature enhancement effect, and not destroying the trunk branch structure of the residual layer in Resnet. This is because the present invention independently adds the SENet module directly between the upper and lower residual layers of Resnet50. The trunk feature extraction network module of the present invention is now specifically described:

[0016] The backbone feature extraction network module consists of a conv1 layer, a conv2_x layer, a first-channel attention mechanism module SENet layer, a conv3_x layer, a second-channel attention mechanism module SENet layer, a conv4_x layer, and a third-channel attention mechanism module SENet layer, through which signals are transmitted in one direction in sequence. The input of the backbone feature extraction network module, i.e., the input of the conv1 layer, is the carbon fiber prepreg image, and the output of the backbone feature extraction network module, i.e., the output of the third-channel attention mechanism module SENet layer, is the feature map P1.

[0017] The additional feature layer extraction module is specially designed based on the constructed backbone feature extraction network module in this paper. It is not included in the existing SSD model. The additional feature layer extraction module is mainly used to generate the required feature maps of different scales and needs to be used in conjunction with the backbone feature extraction network module.

[0018] The additional feature layer extraction module of the present invention is now described in detail:

[0019] The additional feature layer extraction module consists of additional layer 1, additional layer 2, additional layer 3, additional layer 4, and additional layer 5, which have the same structure and transmit signals in one direction in sequence. The input of the additional feature layer extraction module, that is, the input of additional layer 1, is the feature map P1, and the output of the additional feature layer extraction module, that is, the output of additional layer 5, is the feature map C6. The outputs of additional layer 1, additional layer 2, additional layer 3, and additional layer 4 correspond to feature map P2, feature map P3, feature map C4, and feature map C5, respectively.

[0020] The multi-scale fusion long-distance context feature extraction module is original to the present invention, and is intended to enhance the model's accurate recognition and positioning of small-scale defects by fusing shallow detail information with deep semantic information. The three feature maps P1, P2, and P3 obtained by the backbone feature extraction network module and the additional feature layer extraction module are used as the input of the multi-scale fusion long-distance context feature extraction module. In order to make the shallow feature maps also have rich semantic information, the deep feature map information and the shallow feature map information are subjected to top-down layer-by-layer feature interaction. Specifically, the feature maps P1, P2, and P3 are first subjected to convolution with a convolution kernel of 1×1 to achieve unified dimensionality reduction of each layer of channels, and then the deep feature maps P3 and P2 are upsampled in sequence and summed element-by-element with the feature maps of the next layer to obtain rich feature interaction information. In one step, the fused new feature layers are subjected to parallel feature extraction using dilated convolutions with different dilation rates (r=1, 3, 5) to expand the receptive field. Then, the dimensionality reduction of each layer channel is achieved again through convolution with a convolution kernel of 1×1. Finally, the original input image features P1, P2, and P3 are remotely bridged and element-by-element fused to obtain multi-scale fused remote context feature extraction maps C1, C2, and C3 containing rich small-scale information. Combined with the feature maps C4, C5, and C6 obtained from the additional feature layer module, a total of six feature maps C1, C2, C3, C4, C5, and C6 are obtained. Finally, the six feature maps C1, C2, C3, C4, C5, and C6 are input into the detector for target classification and position regression. The defect type and defect location are obtained through the non-maximum suppression method.

[0021] The prior art attempts to fuse shallow detail information with deep semantic information, but the fusion method is different from the present invention. The prior art fusion method mainly upsamples the deep information and adds it to the shallow information element by element to perform the fusion. The fusion method of the present invention has the advantages of a larger receptive field and richer small target defect detail information and semantic information compared to the prior art fusion method. This is because the MFRC module designed in the present invention not only upsamples the deep information and adds it to the shallow information element by element, but also further combines dilated convolution with different expansion rates (r = 1, 3, 5) to expand the receptive field, and based on the residual network idea, it fuses the original shallow feature information element by element, effectively enriching the small target defect detail information and semantic information.

[0022] The multi-scale fusion long-distance context feature extraction module of the present invention is now described in detail:

[0023] The workflow of the multi-scale fusion long-range context feature extraction module is as follows:

[0024] (a) The feature maps P1, P2, and P3 are all reduced to 512 dimensions through convolution with a convolution kernel of 1×1, and the corresponding feature maps P1*, P2*, and P3* are obtained respectively;

[0025] (b) Upsample the feature map P3* to make it the same size as the feature map P2*, and then add it to the feature map P2* element by element to obtain the feature map P2^;

[0026] (c) Upsample the feature map P2^ to make it the same size as the feature map P1*, and then perform element-by-element addition and fusion on it with the feature map P1* to obtain the feature map P1^;

[0027] So far, the fusion of feature information at different scales has been preliminarily achieved;

[0028] (d) Parallel feature extraction is performed on the feature maps X using dilated convolutions with dilation rates r = 1, r = 3, and r = 5. The extracted feature information is then element-wise added and fused, and convolution with a convolution kernel of 1×1 is performed to obtain the feature maps Y; the feature maps X are feature maps P1^, feature maps P2^, and feature maps P3*, and the corresponding feature maps Y are feature maps P1', feature maps P2', and feature maps P3' respectively;

[0029] (e) Adding and fusing the feature map P1 and the feature map P1' element by element to obtain the feature map C1; adding and fusing the feature map P2 and the feature map P2' element by element to obtain the feature map C2; adding and fusing the feature map P3 and the feature map P3' element by element to obtain the feature map C3;

[0030] (f) Inputting feature maps C1, C2, C3, C4, C5, and C6 into the detector for target classification and position regression, and obtaining the defect type and location through the non-maximum suppression method; the detector has the ability to simultaneously complete target positioning and classification at one time, and detects the target by convolving the six feature maps C1, C2, C3, C4, C5, and C6;

[0031] (4) Training the CFP-SSD model;

[0032] Set the loss function and training hyperparameters of the CFP-SSD model, and train the CFP-SSD model by feeding the training set;

[0033] (5) Detection;

[0034] The carbon fiber prepreg images during the production process are collected in real time using image acquisition equipment and input into the trained CFP-SSD model to obtain the defect type and defect location.

[0035] As the preferred technical solution:

[0036] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model described above, in step (1), the image acquisition device is an industrial CCD camera; there are four types of defects, namely, open seams, kinks, splits, and holes; the rectangular boxes are manually annotated using the LabelImg tool; the label data is of the xml file data type; and the carbon fiber prepreg image dataset is in the standard VOC format.

[0037] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (2), the expansion method is: flipping, affine transformation or pixel multiplication of the carbon fiber prepreg image in the carbon fiber prepreg image data set to adjust the brightness, contrast or defect position of the carbon fiber prepreg image, wherein flipping is horizontal or vertical rotation.

[0038] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (2), 80% of the expanded carbon fiber prepreg image dataset is used as a training set, and 20% is used as a test set.

[0039] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (3), the additional layer 1 is composed of a convolution layer I with a convolution kernel size of 1×1 in which signals are transmitted unidirectionally in sequence, a batch normalization layer I, an activation function Relu layer I, a convolution layer II with a convolution kernel size of 3×3, a batch normalization layer II, and an activation function Relu layer II.

[0040] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (3), the detector is a one-stage target detector.

[0041] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model described above, in step (4), the loss function of the CFP-SSD model is set to optimize the detection accuracy of the designed CFP-SSD detection model. Two different calculation errors are used to form the CFP-SSD loss function to adjust the network parameters. The expression of the loss function L(x, c, l, g) of the CFP-SSD model is as follows:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047] In the formula, x is the category matching information of the predicted box; c is the category confidence; l is the location information of the predicted box; g is the location information of the real box; N is the number of matched positive samples; α is the weight coefficient, α = 1; L conf (x,c) represents the category loss; L loc (x, l, g) represents the positioning loss; Pos is the set of positive example boxes The jth category matched to the i-th prior frame is the real frame of P; is the category probability of the real box corresponding to the predicted category of the i-th prior box P; Neg is the set of negative example boxes; is the predicted value of the background class; cx is the central horizontal coordinate of the positive example box; cy is the central vertical coordinate of the positive example box; w is the width of the positive example box; h is the height of the positive example box; To predict the regression parameter corresponding to the i-th positive sample; The regression parameter of the jth real box matched by the positive sample i; smooth L1 (x) is smooth L1 Norm loss function.

[0048] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (4), the training hyperparameters of the CFP-SSD model include batch training size, number of iterations, and model initial learning rate. The batch training size is set to 16, the number of iterations is set to 300, and the model initial learning rate is set to 0.002.

[0049] In the carbon fiber prepreg surface defect detection method based on the CFP-SSD model as described above, in step (2), the original image is compressed into an image of 300×300×3;

[0050] In step (3), the size of the carbon fiber prepreg image is 300×300×3; the size of the output feature map of the conv1 layer is 150×150×64; the size of the output feature map of the conv2_x layer is 75×75×256; the size of the output feature map of the first channel attention mechanism module SENet layer is 75×75×256; the size of the output feature map of the conv3_x layer is 38×38×512; the size of the output feature map of the second channel attention mechanism module SENet layer is 38×38×512; the size of the output feature map of the conv4_x layer is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P2 is 19×19×512; the size of the feature map P3 is 10×10×512; the size of the feature map P4 is 10×10×512; the size of the feature map P5 is 10×10×512; the size of the feature map P6 is 10×10×512; the size of the feature map P7 is 10×10×512; the size of the feature map P8 is 10×10×512; the size of the feature map P9 is 10×10×512; the size of the feature map P1 is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P2 is 19×19×512; the size of the feature map P3 is 10×10×512; the size of the feature map P1 is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P The size of map C4 is 5×5×256; the size of feature map C5 is 3×3×256; the size of feature map C6 is 1×1×256; the size of feature map P1* is 38×38×512; the size of feature map P2* is 19×19×512; the size of feature map P3* is 10×10×512; the size of feature map P2^ is 19×19×512; the size of feature map P1^ is 38×38×512; the size of feature map P1' is 38×38×1024; the size of feature map P2' is 19×19×512; the size of feature map P3' is 10×10×512; the size of feature map C1 is 38×38×1024; the size of feature map C2 is 19×19×512; the size of feature map C3 is 10×10×512.

[0051] Beneficial effects

[0052] In order to better improve the model's ability to accurately identify and locate medium and small-scale splitting and hole defects, the present invention transmits the shallow feature map P1 containing rich detail information and the deep feature maps P2 and P3 containing semantic information as input to the multi-scale fusion long-distance context feature extraction module for shallow and deep information interaction, generating feature maps C1, C2, and C3. Finally, the obtained six feature maps C1, C2, C3, C4, C5, and C6 are sent to the detector to achieve classification and position regression.

[0053] The present invention applies the improved Resnet50 backbone feature extraction network module, additional feature layer extraction module and multi-scale fusion long-distance context feature extraction module to construct a CFP surface defect detection method based on CFP-SSD. This method realizes the effective interaction of shallow and deep information, and realizes high-precision defect detection, identification and positioning. Compared with the traditional SSD detection method, the detection accuracy is significantly improved, especially for the detection of small-scale defects.

[0054] A CFP surface defect detection method based on CFP-SSD designed by the present invention can achieve an average detection accuracy (mAP%) of 86.63% on a self-made CFP dataset, which is 6.03% higher than the detection accuracy of the traditional SSD method; the detection accuracy of the present invention for various types of defects is better than the traditional SSD method, especially for small-scale defects, with the highest improvement of detection capability, which is 10.22%. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the overall network structure of the surface defect detection method of the present invention;

[0056] Figure 2 This is a schematic diagram of the structure of the backbone feature extraction network module in the present invention;

[0057] Figure 3 Schematic diagram of the structure of the additional feature extraction layer in the present invention;

[0058] Figure 4 Schematic diagram of the structure of the multi-scale fusion long-distance context feature extraction module in the present invention. DETAILED DESCRIPTION

[0059] Below in conjunction with specific examples, further set forth the present invention.Should be understood that these examples are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0060] A carbon fiber prepreg surface defect detection method based on the CFP-SSD model includes the following steps:

[0061] (1) Using image acquisition equipment to collect carbon fiber prepreg images in real time during the production process, the defects in the carbon fiber prepreg images were manually marked with rectangular boxes using the LabelImg tool, and corresponding label data was generated for each carbon fiber prepreg image to establish a carbon fiber prepreg image dataset;

[0062] The image acquisition device is an industrial CCD camera;

[0063] There are four types of defects: open seams, kinks, splits, and holes;

[0064] The tag data is of XML file data type;

[0065] The carbon fiber prepreg image dataset is in the standard VOC format;

[0066] (2) Expand the carbon fiber prepreg image dataset, use 80% of the expanded carbon fiber prepreg image dataset as the training set and 20% as the test set (the test set is used to test the accuracy of the trained model in identifying defects, and is mainly used in the comparison results in Table 1 in the following example. The results in Table 1 are obtained on the test set), and compress the original image into a 300×300×3 image;

[0067] The expansion method is as follows: flipping, affine transformation or pixel multiplication of the carbon fiber prepreg image in the carbon fiber prepreg image dataset is performed to adjust the brightness, contrast or defect position of the carbon fiber prepreg image, wherein flipping is horizontal or vertical rotation;

[0068] (3) Constructing the CFP-SSD model;

[0069] like Figures 1 to 4 As shown in Figure 2, the CFP-SSD model consists of a backbone feature extraction network module, an additional feature layer extraction module, and a multi-scale fusion long-distance context feature extraction module.

[0070] like Figure 2 As shown, the backbone feature extraction network module consists of a conv1 layer, a conv2_x layer, a first channel attention mechanism module SENet layer, a conv3_x layer, a second channel attention mechanism module SENet layer, a conv4_x layer, and a third channel attention mechanism module SENet layer, in which signals are transmitted in one direction in sequence; the input of the backbone feature extraction network module, i.e., the input of the conv1 layer, is the carbon fiber prepreg image, and the output of the backbone feature extraction network module, i.e., the output of the third channel attention mechanism module SENet layer, is the feature map P1;

[0071] The carbon fiber prepreg image size is 300×300×3;

[0072] The output feature map size of the conv1 layer is 150×150×64;

[0073] The output feature map size of the conv2_x layer is 75×75×256;

[0074] The output feature map size of the SENet layer of the first channel attention mechanism module is 75×75×256;

[0075] The output feature map size of the conv3_x layer is 38×38×512;

[0076] The output feature map size of the SENet layer of the second channel attention mechanism module is 38×38×512;

[0077] The output feature map size of the conv4_x layer is 38×38×1024;

[0078] The size of feature map P1 is 38×38×1024;

[0079] like Figure 3 As shown, the additional feature layer extraction module consists of additional layer 1, additional layer 2, additional layer 3, additional layer 4, and additional layer 5 with the same structure and unidirectional signal transmission in sequence; the input of the additional feature layer extraction module, that is, the input of the additional layer 1, is the feature map P1, and the output of the additional feature layer extraction module, that is, the output of the additional layer 5, is the feature map C6; the outputs of the additional layer 1, additional layer 2, additional layer 3, and additional layer 4 correspond to feature map P2, feature map P3, feature map C4, and feature map C5, respectively;

[0080] The extra layer 1 consists of a convolution layer I with a convolution kernel size of 1×1, a batch normalization layer I, an activation function ReLU layer I, a convolution layer II with a convolution kernel size of 3×3, a batch normalization layer II, and an activation function ReLU layer II.

[0081] The size of feature map P2 is 19×19×512; the size of feature map P3 is 10×10×512; the size of feature map C4 is 5×5×256; the size of feature map C5 is 3×3×256; the size of feature map C6 is 1×1×256;

[0082] like Figure 4 As shown in Figure 2, the workflow of the multi-scale fusion long-distance context feature extraction module is as follows:

[0083] (a) The feature maps P1, P2, and P3 are all reduced to 512 dimensions by convolution with a convolution kernel of 1×1, resulting in feature maps P1* of size 38×38×512, P2* of size 19×19×512, and P3* of size 10×10×512, respectively.

[0084] (b) Upsample the feature map P3* to make it the same size as the feature map P2*, and then add it to the feature map P2* element by element to obtain the feature map P2^ of size 19×19×512;

[0085] (c) Upsample the feature map P2^ to make it the same size as the feature map P1*, and then perform element-by-element addition and fusion with the feature map P1* to obtain a feature map P1^ of size 38×38×512;

[0086] (d) Parallel feature extraction is performed on the feature map X using dilated convolution with expansion rates r = 1, r = 3, and r = 5. The extracted feature information is then element-wise added and fused, and convolution with a convolution kernel of 1×1 is used to obtain the feature map Y; the feature map X is the feature map P1^, the feature map P2^, and the feature map P3*. The corresponding feature maps Y are the feature map P1' of size 38×38×1024, the feature map P2' of size 19×19×512, and the feature map P3' of size 10×10×512, respectively.

[0087] (e) The feature maps P1 and P1' are fused element by element to obtain a feature map C1 of size 38 × 38 × 1024; the feature maps P2 and P2' are fused element by element to obtain a feature map C2 of size 19 × 19 × 512; the feature maps P3 and P3' are fused element by element to obtain a feature map C3 of size 10 × 10 × 512;

[0088] (f) Inputting feature maps C1, C2, C3, C4, C5, and C6 into a detector (one-stage target detector) for target classification and position regression, and obtaining the defect type and location through a non-maximum suppression method. The detector has the ability to simultaneously complete target positioning and classification at one time, and detects the target by performing convolution on the six feature maps C1, C2, C3, C4, C5, and C6.

[0089] (4) Training the CFP-SSD model;

[0090] Set the loss function and training hyperparameters of the CFP-SSD model, and train the CFP-SSD model by feeding the training set;

[0091] The loss function L(x,c,l,g) of the CFP-SSD model is expressed as follows:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097] In the formula, x is the category matching information of the predicted box; c is the category confidence; l is the location information of the predicted box; g is the location information of the real box; N is the number of matched positive samples; α is the weight coefficient, α = 1; L conf(x,c) represents the category loss; L loc (x, l, g) represents the positioning loss; Pos is the set of positive example boxes; The jth category matched to the i-th prior frame is the real frame of P; is the category probability of the real box corresponding to the predicted category of the i-th prior box P; Neg is the set of negative example boxes; is the predicted value of the background class; cx is the central horizontal coordinate of the positive example box; cy is the central vertical coordinate of the positive example box; w is the width of the positive example box; h is the height of the positive example box; To predict the regression parameter corresponding to the i-th positive sample; The regression parameter of the jth real box matched by the positive sample i; smooth L1 (x) is smooth L1 Norm loss function;

[0098] The training hyperparameters of the CFP-SSD model include batch training size, number of iterations, and model initial learning rate. The batch training size is set to 16, the number of iterations is set to 300, and the model initial learning rate is set to 0.002.

[0099] (5) Detection;

[0100] The carbon fiber prepreg images during the production process are collected in real time using image acquisition equipment and input into the trained CFP-SSD model to obtain the defect type and defect location.

[0101] Experiments were conducted on a self-made CFP test set using the traditional SSD method and the carbon fiber prepreg surface defect detection method based on the CFP-SSD model of the present invention. The experimental results are shown in Table 1.

[0102] Table 1

[0103] Model Input resolution open gap Kink Splitting holes mAP% SSD 300×300 78.95％ 92.2％ 83.91％ 67.34％ 80.60％ CFP-SSD 300×300 84.58％ 95.56％ 88.8％ 77.56％ 86.63％

[0104] From the data in Table 1, it can be seen that the carbon fiber prepreg surface defect detection method based on the CFP-SSD model of the present invention can achieve an average detection accuracy (mAP%) of 86.63% on the homemade CFP test set, which is 6.03% higher than the average detection accuracy of the traditional SSD method. It can also be seen that the traditional SSD method has poor detection effect on smaller defects (holes and gaps in Table 1), while the detection accuracy of the present invention is better than the traditional SSD method for various defects, especially for small-scale defects, where the detection accuracy is greatly improved.

[0105] The above is a schematic description of the present invention and its implementation methods. This description is not restrictive. If ordinary technicians in this field are inspired by it and design methods and embodiments similar to the technical solution without creatively designing them without departing from the purpose of the invention, they should all fall within the scope of protection of the invention.

Claims

1. A carbon fiber prepreg surface defect detection method based on the CFP-SSD model, characterized in that: The following steps are involved: (1) Using image acquisition equipment to collect carbon fiber prepreg images in real time during the production process, manually mark the defects in the carbon fiber prepreg images with rectangular frames, generate corresponding label data for each carbon fiber prepreg image, and establish a carbon fiber prepreg image dataset; (2) Expanding the carbon fiber prepreg image dataset, using a portion of the expanded carbon fiber prepreg image dataset as a training set, and compressing the original images; (3) Constructing the CFP-SSD model; The CFP-SSD model consists of a backbone feature extraction network module, an additional feature layer extraction module, and a multi-scale fusion long-distance context feature extraction module; The backbone feature extraction network module consists of a conv1 layer, a conv2_x layer, a first-channel attention mechanism module SENet layer, a conv3_x layer, a second-channel attention mechanism module SENet layer, a conv4_x layer, and a third-channel attention mechanism module SENet layer, through which signals are transmitted in one direction in sequence. The input of the backbone feature extraction network module, i.e., the input of the conv1 layer, is the carbon fiber prepreg image, and the output of the backbone feature extraction network module, i.e., the output of the third-channel attention mechanism module SENet layer, is the feature map P1. The additional feature layer extraction module consists of additional layer 1, additional layer 2, additional layer 3, additional layer 4, and additional layer 5, which have the same structure and signal transmission in one direction. The input of the additional feature layer extraction module, i.e., the input of additional layer 1, is feature map P1, and the output of the additional feature layer extraction module, i.e., the output of additional layer 5, is feature map C6; the outputs of additional layer 1, additional layer 2, additional layer 3, and additional layer 4 correspond to feature map P2, feature map P3, feature map C4, and feature map C5, respectively; The workflow of the multi-scale fusion long-range context feature extraction module is as follows: (a) The feature maps P1, P2, and P3 are all reduced to 512 dimensions through convolution with a convolution kernel of 1×1, and the corresponding feature maps P1*, P2*, and P3* are obtained respectively; (b) Upsample the feature map P3* to make it the same size as the feature map P2*, and then add it to the feature map P2* element by element to obtain the feature map P2^; (c) Upsample the feature map P2^ to make it the same size as the feature map P1*, and then perform element-by-element addition and fusion on it with the feature map P1* to obtain the feature map P1^; (d) Parallel feature extraction is performed on the feature maps X using dilated convolutions with dilation rates r = 1, r = 3, and r = 5. The extracted feature information is then element-wise added and fused, and convolution with a convolution kernel of 1×1 is performed to obtain the feature maps Y; the feature maps X are feature maps P1^, feature maps P2^, and feature maps P3*, and the corresponding feature maps Y are feature maps P1', feature maps P2', and feature maps P3' respectively; (e) Adding and fusing the feature map P1 and the feature map P1' element by element to obtain the feature map C1; adding and fusing the feature map P2 and the feature map P2' element by element to obtain the feature map C2; adding and fusing the feature map P3 and the feature map P3' element by element to obtain the feature map C3; (f) Inputting feature maps C1, C2, C3, C4, C5, and C6 into the detector for target classification and position regression, and obtaining the defect type and defect location through the non-maximum suppression method; (4) Training the CFP-SSD model; Set the loss function and training hyperparameters of the CFP-SSD model, and train the CFP-SSD model by feeding the training set; (5) Detection; The carbon fiber prepreg images during the production process are collected in real time using image acquisition equipment and input into the trained CFP-SSD model to obtain the defect type and defect location.

2. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (1), the image acquisition device is an industrial CCD camera; there are four types of defects, namely open seams, kinks, splits, and holes; the LabelImg tool is used to manually mark rectangular boxes; the label data is of the XML file data type; and the carbon fiber prepreg image dataset is in the standard VOC format.

3. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (2), the expansion method is: flipping, affine transformation or pixel multiplication of the carbon fiber prepreg image in the carbon fiber prepreg image data set to adjust the brightness, contrast or defect position of the carbon fiber prepreg image, wherein flipping is horizontal or vertical rotation.

4. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (2), 80% of the expanded carbon fiber prepreg image dataset is used as a training set, and 20% is used as a test set.

5. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (3), the additional layer 1 consists of a convolution layer I with a convolution kernel size of 1×1, a batch normalization layer I, an activation function ReLU layer I, a convolution layer II with a convolution kernel size of 3×3, a batch normalization layer II, and an activation function ReLU layer II, in which signals are transmitted unidirectionally.

6. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (3), the detector is a one-stage object detector.

7. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (4), the loss function L(x,c,l,g) of the CFP-SSD model is expressed as follows: In the formula, x is the category matching information of the predicted box; c is the category confidence; l is the location information of the predicted box; g is the location information of the real box; N is the number of matched positive samples; α is the weight coefficient, α = 1; L conf (x,c) represents the category loss; L loc (x, l, g) represents the positioning loss; Pos is the set of positive example boxes; The jth category matched to the i-th prior frame is the real frame of P; is the category probability of the real box corresponding to the predicted category of the i-th prior box P; Neg is the set of negative example boxes; is the predicted value of the background class; cx is the central horizontal coordinate of the positive example box; cy is the central vertical coordinate of the positive example box; w is the width of the positive example box; h is the height of the positive example box; To predict the regression parameter corresponding to the i-th positive sample; The regression parameter of the jth real box matched by the positive sample i; smooth L1 (x) is smooth L1 Norm loss function.

8. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (4), the training hyperparameters of the CFP-SSD model include batch training size, number of iterations, and model initial learning rate. The batch training size is set to 16, the number of iterations is set to 300, and the model initial learning rate is set to 0.

002.

9. The carbon fiber prepreg surface defect detection method based on the CFP-SSD model according to claim 1, characterized in that: In step (2), the original image is compressed into a 300×300×3 image; In step (3), the size of the carbon fiber prepreg image is 300×300×3; the size of the output feature map of the conv1 layer is 150×150×64; the size of the output feature map of the conv2_x layer is 75×75×256; the size of the output feature map of the first channel attention mechanism module SENet layer is 75×75×256; the size of the output feature map of the conv3_x layer is 38×38×512; the size of the output feature map of the second channel attention mechanism module SENet layer is 38×38×512; the size of the output feature map of the conv4_x layer is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P2 is 19×19×512; the size of the feature map P3 is 10×10×512; the size of the feature map P4 is 10×10×512; the size of the feature map P5 is 10×10×512; the size of the feature map P6 is 10×10×512; the size of the feature map P7 is 10×10×512; the size of the feature map P8 is 10×10×512; the size of the feature map P9 is 10×10×512; the size of the feature map P1 is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P2 is 19×19×512; the size of the feature map P3 is 10×10×512; the size of the feature map P1 is 38×38×1024; the size of the feature map P1 is 38×38×1024; the size of the feature map P The size of map C4 is 5×5×256; the size of feature map C5 is 3×3×256; the size of feature map C6 is 1×1×256; the size of feature map P1* is 38×38×512; the size of feature map P2* is 19×19×512; the size of feature map P3* is 10×10×512; the size of feature map P2^ is 19×19×512; the size of feature map P1^ is 38×38×512; the size of feature map P1' is 38×38×1024; the size of feature map P2' is 19×19×512; the size of feature map P3' is 10×10×512; the size of feature map C1 is 38×38×1024; the size of feature map C2 is 19×19×512; the size of feature map C3 is 10×10×512.

Images