High-precision prepreg defect detection method based on industrial camera

Abstract

The invention relates to the technical field of industrial detection, in particular to a high-precision prepreg defect detection method based on an industrial camera, which comprises the following steps: acquiring a glass fiber prepreg surface image; the glass fiber prepreg surface image is input into a preset defect detection model, a glass fiber prepreg surface defect detection result is output, the defect detection model is obtained based on training of a training set, the training set comprises the glass fiber prepreg surface image marked with various defects, and the defect detection result is obtained based on the defect detection result. And the defect detection model is used for performing multi-layer feature extraction and feature fusion on the surface image of the glass fiber prepreg cloth and predicting the defect position and category. According to the method, prepreg defect detection is carried out by introducing the improved multi-scale convolution mechanism and the grouping cascade attention improved RT-DETR model, and the small target detection effect can be improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection technology, and in particular to a high-precision prepreg defect detection method based on an industrial camera. Background Art

[0002] As industrial production continues to demand higher quality products, the detection of surface defects in glass fiber prepreg, a key component of composite materials, plays a crucial role in product performance. However, due to the small size, complex shape, and low contrast of some surface defects in glass fiber prepreg (such as impurity bubbles and cracks), traditional manual inspection and simple machine vision methods struggle to meet the required accuracy and efficiency.

[0003] In recent years, deep learning has made significant progress in the field of target detection, and its application in surface defect detection has gradually become a research hotspot. In particular, the RT-DETR model has received widespread attention in the field of industrial detection due to its high efficiency and accuracy. The RT-DETR model achieves end-to-end target detection by introducing the Transformer mechanism, which improves detection efficiency. However, it has some technical problems: (1) The ability to detect small targets is insufficient, especially for small-sized defects such as impurities and bubbles, which are difficult for existing models to accurately locate and classify. (2) The feature map generation part has insufficient ability to capture multi-scale features and there is computational redundancy, which affects the real-time detection performance of the model. (3) The attention mechanism has limitations when focusing on small targets and fails to fully improve the detection effect of small targets. Summary of the Invention

[0004] The purpose of the present invention is to address the above technical problems and provide a high-precision prepreg defect detection method based on industrial cameras. The method optimizes feature map generation by introducing an improved multi-scale convolution mechanism, and improves the attention mechanism in AIFI (Anchor-Free Instance-Focus) by combining grouped cascade attention to improve the small target detection performance.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] High-precision prepreg defect detection method based on industrial cameras, including:

[0007] Acquire the surface image of glass fiber prepreg;

[0008] The glass fiber prepreg surface image is input into a preset defect detection model, and a glass fiber prepreg surface defect detection result is output, wherein the defect detection model is obtained based on training of a training set, and the training set includes glass fiber prepreg surface images marked with various defects. The defect detection model is used to perform multi-layer feature extraction and feature fusion on the glass fiber prepreg surface image to predict the defect location and category.

[0009] Optionally, the defect detection model includes:

[0010] A multi-scale feature extraction module is used to extract multi-scale features from the input glass fiber prepreg surface image to obtain a multi-scale feature map;

[0011] A high-level feature extraction module, configured to perform improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map;

[0012] A feature interaction module, configured to perform feature interaction on the high-level feature map using a grouped cascade attention mechanism to obtain an enhanced feature map;

[0013] A feature fusion module is used to perform cross-scale feature fusion on the multi-scale feature map and the enhanced feature map to obtain a fused feature map;

[0014] The defect detection module is used to predict the defect location and category based on the fused feature map, and output it after annotation.

[0015] Optionally, the multi-scale feature extraction module performs multi-scale feature extraction on the input glass fiber prepreg surface image to obtain a multi-scale feature map, including:

[0016] The hierarchical structure of the ResNet-18 network is used to extract multi-scale features from the input surface image of the glass fiber prepreg, and output feature maps of different sizes generated from shallow to deep as the network depth increases.

[0017] Optionally, the high-level feature extraction module performs improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map, including:

[0018] Performing regional residual processing on the image containing high-level features in the multi-scale feature map to obtain a plurality of regional feature maps;

[0019] Performing semantic residual processing on the regional feature map using a preset receptive field to obtain multi-scale features and then fusing them to obtain an expanded feature map;

[0020] All expanded feature maps are fused into the final output feature map, i.e., the high-level feature map.

[0021] Optionally, performing regional residual processing on the image containing high-level features in the multi-scale feature map includes:

[0022] F RR =ReLU(BN(W1*F in ));

[0023] Among them, F RR is the output feature map after regional residualization, F inis the input feature map, W1 is the regional residual convolution kernel weight, BN is the batch normalization operation, and ReLU is the activation function.

[0024] Optionally, performing semantic residual processing on the regional feature map using a preset receptive field includes:

[0025] F SR,d =W d ⊙F RR ;

[0026] Among them, W d is the weight of the deep dilated convolution kernel with dilation rate d, ⊙ is the deep dilated convolution operation, F SR,d It is the feature map after semantic residualization.

[0027] Optionally, the feature interaction module uses a grouped cascade attention mechanism to perform feature interaction on the high-level feature map, and obtaining the enhanced feature map includes:

[0028] Divide the high-level feature map into several sub-feature maps, perform single-head attention calculation on each sub-feature map, obtain the output features of several single heads and integrate them, and finally extract the local and global feature relationship to obtain the enhanced feature map;

[0029] Among them, in the single-head attention calculation process, starting from the second single head, the output features of the previous single head are added to the input features of the current single head.

[0030] Optionally, the feature fusion module adopts the cross-channel feature fusion structure of the RT-DETR model to perform cross-scale feature fusion processing on the image containing the middle-level features in the multi-scale feature map and the enhanced feature map;

[0031] The defect detection module uses the decoder of the RT-DETR model to predict the defect location and category, mark the defect location and indicate the defect type and confidence level.

[0032] The beneficial effects of the present invention are:

[0033] (1) Small target detection accuracy is significantly improved: By improving the multi-scale convolution mechanism and introducing the cascade group attention mechanism, the present invention significantly improves the detection accuracy of small targets such as impurity bubbles and defect detection accuracy.

[0034] (2) Enhanced feature extraction capability: The improved feature map generation mechanism of the present invention effectively enhances the expression capability of multi-scale features and improves the adaptability of the model to complex backgrounds.

[0035] (3) Strong real-time performance: Through lightweight optimization, the present invention improves the detection speed while ensuring high accuracy of the model, meeting the requirements of real-time detection of industrial production lines.

[0036] (4) High versatility: The present invention is not only applicable to the detection of surface defects of glass fiber prepreg, but can also be extended to other industrial defect detection tasks, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flow chart of a high-precision prepreg defect detection method based on an industrial camera according to an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of an improved multi-scale convolution process according to an embodiment of the present invention;

[0040] Figure 3 This is a flowchart of the grouped cascade attention mechanism processing according to an embodiment of the present invention;

[0041] Figure 4 A comparison chart of the test results of the present model and the comparison model for Test 1 according to an embodiment of the present invention;

[0042] Figure 5 This is a comparison chart of the detection results of the present model and the comparison model for Test 2 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] This embodiment provides a high-precision prepreg defect detection method based on an industrial camera, including:

[0046] Acquire the surface image of glass fiber prepreg;

[0047] The glass fiber prepreg surface image is input into a preset defect detection model, and a glass fiber prepreg surface defect detection result is output, wherein the defect detection model is obtained based on training of a training set, and the training set includes glass fiber prepreg surface images marked with various defects. The defect detection model is used to perform multi-layer feature extraction and feature fusion on the glass fiber prepreg surface image to predict the defect location and category.

[0048] Specifically, this embodiment introduces an improved multi-scale convolution mechanism and a grouped cascade attention improved RT-DETR model to construct a defect detection model, which significantly improves the detection accuracy of small targets such as impurities and bubbles and the defect detection accuracy. Figure 1 As shown, the construction and workflow of the defect detection model in this embodiment specifically include:

[0049] (1) Read the image and organize the images containing various types of defects generated during the production process of glass fiber prepreg into a training set for subsequent model training. The specific process is as follows:

[0050] Camera selection and deployment: To minimize the impact of uneven lighting and improve image acquisition quality, high-resolution line scan industrial cameras were deployed at the reeling position of the fiberglass prepreg production line, ensuring coverage of the entire wide area of ​​the fabric and effectively capturing complete surface images.

[0051] Image Acquisition: An industrial camera, coupled with camera driver software, is linked to an encoder for control. Continuous image acquisition is performed based on the set fabric movement length as the trigger interval, ensuring the continuity and representativeness of image data, making it suitable for real-time defect detection.

[0052] Defect sample capture and labeling: During the image acquisition process, images containing typical defects (such as impurities, bubbles, yellowing, broken yarn, etc.) are manually labeled and classified to build a structured training dataset, providing high-quality sample support for subsequent deep learning model training.

[0053] (2) Model construction, the defect detection model includes:

[0054] A multi-scale feature extraction module is used to extract multi-scale features from the input glass fiber prepreg surface image to obtain a multi-scale feature map;

[0055] A high-level feature extraction module, configured to perform improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map;

[0056] A feature interaction module, configured to perform feature interaction on the high-level feature map using a grouped cascade attention mechanism to obtain an enhanced feature map;

[0057] A feature fusion module is used to perform cross-scale feature fusion on the multi-scale feature map and the enhanced feature map to obtain a fused feature map;

[0058] The defect detection module is used to predict the defect location and category based on the fused feature map, and output it after annotation.

[0059] Furthermore, the multi-scale feature extraction module performs multi-scale feature extraction on the input glass fiber prepreg surface image, and obtains a multi-scale feature map including:

[0060] The hierarchical structure of the ResNet-18 network is used to extract multi-scale features from the input surface image of the glass fiber prepreg, and output feature maps of different sizes generated from shallow to deep as the network depth increases.

[0061] Specifically, compared to ResNet-50 and ResNet-101, ResNet-18 has a shallower network and higher computational efficiency, making it suitable for scenarios requiring fast inference. The layered structure of ResNet-18 is used to extract the multi-scale feature map of the defect image from the previous step. The convolutional layer formula is:

[0062]

[0063] Among them, K is the convolution kernel size, W m,n,c,k is the convolution kernel weight, b k is the bias.

[0064] The output feature map size is:

[0065]

[0066] Where P is the padding size and S is the stride.

[0067] The maximum pooling formula is:

[0068]

[0069] The spatial size of the output feature map gradually decreases with the depth of the network, and finally generates five feature maps of different sizes: P1, P2, P3, P4, and P5 according to the network from shallow to deep.

[0070] Furthermore, the high-level feature extraction module performs improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map, including:

[0071] Performing regional residual processing on the image containing high-level features in the multi-scale feature map to obtain a plurality of regional feature maps;

[0072] Performing semantic residual processing on the regional feature map using a preset receptive field to obtain multi-scale features and then fusing them to obtain an expanded feature map;

[0073] All expanded feature maps are fused into the final output feature map, i.e., the high-level feature map.

[0074] Specifically, the high-level feature map P5 generated in the previous step is used as the input feature map F in Improve multi-scale convolution, the processing flow is as follows Figure 2 As shown, the following steps are included:

[0075] ① In the regional residualization stage, the original feature map is processed according to the expected receptive field to generate a series of concise and targeted regional feature maps. These feature maps are initially separated according to different receptive field requirements to reduce redundancy and noise. The formula is:

[0076] F RR =ReLU(BN(W1*F in ));

[0077] Among them, F RR is the output feature map after regional residualization, F in is the input feature map, W1 is the regional residual convolution kernel weight, BN is the batch normalization operation, and ReLU is the activation function.

[0078] ② In the semantic residualization stage, the regional feature maps generated in the first stage are convolved with the corresponding receptive fields, and then the extracted multi-scale features are fused to obtain effective multi-scale context information. RR , the semantic residual operation is:

[0079] F SR,d =W d ⊙F RR ;

[0080] Among them, W d is the weight of the deep dilated convolution kernel with dilation rate d, ⊙ is the deep dilated convolution operation, F SR,d It is a semantic residual feature map with context information corresponding to the receptive field.

[0081] ③ After semantic residualization, all expansion rates d∈{d1,d2,...,d n The generated feature maps are fused, and the final output feature map after fusion is:

[0082]

[0083] Among them, Fusion is a feature fusion operation.

[0084] Furthermore, the feature interaction module uses a grouped cascade attention mechanism to perform feature interaction on the high-level feature map, and obtaining an enhanced feature map includes:

[0085] Divide the high-level feature map into several sub-feature maps, perform single-head attention calculation on each sub-feature map, obtain the output features of several single heads and integrate them, and finally extract the local and global feature relationship to obtain the enhanced feature map;

[0086] Among them, in the single-head attention calculation process, starting from the second single head, the output features of the previous single head are added to the input features of the current single head.

[0087] Specifically, for the F generated in the third step out As input feature X i The grouped cascade attention mechanism is used for feature interaction. Compared with the traditional multi-head attention mechanism, the grouped cascade attention mechanism has better interpretability and can reduce computational redundancy and improve the real-time performance of the model. The processing flow is as follows: Figure 3 As shown, the following steps are included:

[0088] ①In the feature division stage, the input feature X i ∈R C×H×W Divide into h sub-feature blocks by channel:

[0089] X i =[X i1 ,X i2 ,...,X ih ];

[0090] Among them, h is the number of groups, X ij is the input sub-feature block of the j-th head.

[0091] ②In the single-head attention calculation stage, each head calculates the self-attention of the sub-feature block separately:

[0092]

[0093] in, They are the Query, Key, and Value projection matrices of the j-head respectively. is the output feature of the j-th head.

[0094] ③In the cascade enhancement stage, starting from the second head, the output features of the previous head are Add to the input features of the current head to gradually enhance the representation capability:

[0095]

[0096] Among them, X′ ijAs the new input feature of the jth head. For the first head, directly use the initial grouping feature X i1 .

[0097] ④In the feature integration stage, the output features of all heads are Concatenate by channel dimension:

[0098]

[0099] in, is a linear projection matrix used to map the concatenated features back to the original input dimension.

[0100] ⑤After Q-projection, an additional Token Interaction Layer is used to capture local and global feature relationships and further enhance the representation capability.

[0101] Furthermore, the feature fusion module adopts the cross-channel feature fusion structure of the RT-DETR model to perform cross-scale feature fusion processing on the image containing mid-level features in the multi-scale feature map and the enhanced feature map.

[0102] Specifically, the cross-channel feature fusion structure (CCFF) of the RTDETR model is used to take the output of the fourth step and P3 and P4 of the second step as input for cross-scale feature fusion, ensuring that the final feature map has information of multiple resolutions.

[0103] Furthermore, the defect detection module uses the decoder of the RT-DETR model to predict the defect location and category, mark the defect location and indicate the defect type and confidence level.

[0104] Specifically, the decoder of the RTDETR model is used to predict the object location and category. For prepreg images that are detected to have defects, the defect location is marked and the defect type and confidence level are indicated. The marked images are then output.

[0105] (3) Model test: This model is compared with the RT-DETR benchmark model. Two images of the glass fiber prepreg surface are input as test 1 and test 2 respectively. The test results are as follows: Figure 4 、 Figure 5 As shown in the figure, the left side is the detection result of this model, and the right side is the detection result of the comparison model. This model improves the detection ability of small targets while ensuring the detection accuracy of the baseline model.

[0106] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A high-precision prepreg defect detection method based on an industrial camera, characterized in that: include: Acquire the surface image of glass fiber prepreg; The glass fiber prepreg surface image is input into a preset defect detection model, and a glass fiber prepreg surface defect detection result is output, wherein the defect detection model is obtained based on training of a training set, and the training set includes glass fiber prepreg surface images marked with various defects. The defect detection model is used to perform multi-layer feature extraction and feature fusion on the glass fiber prepreg surface image to predict the defect location and category.

2. The high-precision prepreg defect detection method based on an industrial camera according to claim 1, characterized in that: The defect detection model includes: A multi-scale feature extraction module is used to extract multi-scale features from the input glass fiber prepreg surface image to obtain a multi-scale feature map; A high-level feature extraction module, configured to perform improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map; A feature interaction module, configured to perform feature interaction on the high-level feature map using a grouped cascade attention mechanism to obtain an enhanced feature map; A feature fusion module is used to perform cross-scale feature fusion on the multi-scale feature map and the enhanced feature map to obtain a fused feature map; The defect detection module is used to predict the defect location and category based on the fused feature map, and output it after annotation.

3. The high-precision prepreg defect detection method based on an industrial camera according to claim 2, characterized in that: The multi-scale feature extraction module extracts multi-scale features from the input glass fiber prepreg surface image to obtain a multi-scale feature map, including: The hierarchical structure of the ResNet-18 network is used to extract multi-scale features from the input surface image of the glass fiber prepreg, and output feature maps of different sizes generated from shallow to deep as the network depth increases.

4. The high-precision prepreg defect detection method based on an industrial camera according to claim 2, characterized in that: The high-level feature extraction module performs improved multi-scale convolution processing on the multi-scale feature map to obtain a high-level feature map, including: Performing regional residual processing on the image containing high-level features in the multi-scale feature map to obtain a plurality of regional feature maps; Performing semantic residual processing on the regional feature map using a preset receptive field to obtain multi-scale features and then fusing them to obtain an expanded feature map; All expanded feature maps are fused into the final output feature map, i.e., the high-level feature map.

5. The high-precision prepreg defect detection method based on an industrial camera according to claim 4, characterized in that: Performing regional residual processing on the image containing high-level features in the multi-scale feature map includes: F RR =ReLU(BN(W1*F in )); Among them, F RR is the output feature map after regional residualization, F in is the input feature map, W1 is the regional residual convolution kernel weight, BN is the batch normalization operation, and ReLU is the activation function.

6. The high-precision prepreg defect detection method based on an industrial camera according to claim 4, characterized in that: Performing semantic residual processing on the regional feature map using a preset receptive field includes: F SR,d =W d ⊙F RR ; Among them, W d is the weight of the deep dilated convolution kernel with dilation rate d, ⊙ is the deep dilated convolution operation, F SR,d It is the feature map after semantic residualization.

7. The high-precision prepreg defect detection method based on an industrial camera according to claim 2, characterized in that: The feature interaction module uses a grouped cascade attention mechanism to perform feature interaction on the high-level feature map, and obtaining an enhanced feature map includes: Divide the high-level feature map into several sub-feature maps, perform single-head attention calculation on each sub-feature map, obtain the output features of several single heads and integrate them, and finally extract the local and global feature relationship to obtain the enhanced feature map; Among them, in the single-head attention calculation process, starting from the second single head, the output features of the previous single head are added to the input features of the current single head.

8. The high-precision prepreg defect detection method based on an industrial camera according to claim 2, characterized in that: The feature fusion module uses the cross-channel feature fusion structure of the RT-DETR model to perform cross-scale feature fusion processing on the image containing the middle-level features in the multi-scale feature map and the enhanced feature map; The defect detection module uses the decoder of the RT-DETR model to predict the defect location and category, mark the defect location and indicate the defect type and confidence level.

Images