A workpiece defect classification method and system based on a convolution-Transformer fusion network

Abstract

The application relates to the technical field of deep learning and computer vision, and discloses a workpiece defect classification method and system based on a convolution-Transformer fusion network, which comprises the following steps: acquiring and preprocessing a workpiece surface defect image; constructing a fusion network; inputting the preprocessed image into a convolution coding module to obtain a plurality of dimension characteristic vectors; inputting the characteristic vectors into a local aggregation module to obtain a first characteristic vector; inputting the aggregated first characteristic vector into a global-local sparse attention module to obtain a feature saliency map; inputting the saliency map into the guiding aggregation module to obtain a second characteristic vector; inputting the aggregated second characteristic vector into a down-sampling layer to obtain a sampled characteristic vector; obtaining a final characteristic vector; inputting the final characteristic vector into a double-layer perception machine output module to obtain a classification result. The application can improve the defect classification accuracy and reduce the defect classification calculation amount.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision technology, and in particular to a method and system for classifying workpiece defects based on a convolutional-transformer fusion network. Background Technology

[0002] Visual inspection technology for surface defects is crucial for achieving intelligent manufacturing. In defect detection, defect classification effectively distinguishes and identifies different types of defects, providing guidance for subsequent repair and processing. Accurate defect classification enables timely identification of different defect types, allowing for targeted measures to improve quality and efficiency in manufacturing and service sectors. Correct defect classification helps reduce false alarms and false negatives, improves the reliability of the inspection system, and provides enterprises with more reliable quality control methods. However, existing neural network-based defect classification methods suffer from low accuracy and high computational cost.

[0003] Existing technology discloses an intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer (MAViT), including a data preprocessing module, an attention-based spatial transformation module, an adversarial perturbation module, a VIT module, and a classifier module. The attention-based coordinate space transformation module includes a multi-spectral channel attention unit and a coordinated attention unit. MAViT is an attention-based network model suitable for computer vision, capable of performing image classification tasks well. Even with a limited number of defective industrial images, it can quickly and stably evaluate the surface morphology features of industrial materials such as composite fibers. This model utilizes an attention mechanism, a technique in deep learning models used to enhance the focus on and utilization of important information. It can be seen as a way to simulate human attention mechanisms, enabling the model to concentrate on specific parts or features when processing input data. This system uses the VIT module to segment images, which easily destroys some image features, leading to inaccurate classification. Furthermore, it uses the traditional Transformer model for classification, resulting in a large computational load. Summary of the Invention

[0004] The primary objective of this invention is to overcome the problems existing in the prior art and provide a workpiece defect classification method and system based on convolutional-Transformer fusion networks. This invention can improve the accuracy of defect classification while reducing the computational load of defect classification.

[0005] As another objective of the present invention, a system adapted to the method based on the aforementioned objective is also provided.

[0006] To achieve the above objectives, this invention provides a workpiece defect classification method based on a convolutional-Transformer fusion network, comprising:

[0007] Step S1: Obtain images of surface defects on the workpiece to be classified and preprocess the images;

[0008] Step S2: Construct a fusion network by inputting the preprocessed image into the fusion network to obtain defect classification results, specifically including:

[0009] Step S2.1: Construct a fusion network, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction module includes a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module.

[0010] Step S2.2: Input the preprocessed image into the convolutional coding module to obtain feature vectors of multiple dimensions. The convolutional coding module calculates the feature vectors using the following formula:

[0011]

[0012] Where BN represents a normalization layer, and Conv represents a two-dimensional convolution computation. For input data, To output the feature vector;

[0013] Step S2.3: Input the feature vector into the local aggregation module to obtain the first feature vector after aggregation;

[0014] Step S2.4: Input the aggregated first feature vector into the global-local sparse attention module to obtain the feature saliency map;

[0015] Step S2.5: Input the saliency map into the guided aggregation module to obtain the aggregated second feature vector, specifically including:

[0016] Step S2.5.1: Divide the feature saliency map into n sub-regions on average;

[0017] Step S2.5.2: Divide the patch in the input feature vector into n feature vectors according to the n sub-regions;

[0018] Step S2.5.3: Set the corresponding aggregation rate based on the salience of the sub-region;

[0019] Step S2.5.4: Aggregate the n feature vectors using the aggregation rate to obtain multiple sets of aggregated feature vectors;

[0020] Step S2.5.5: Concatenate the aggregated feature vectors to obtain the aggregated second feature vector;

[0021] Step S2.6: Input the aggregated second feature vector into the downsampling layer to obtain the sampled feature vector;

[0022] Step S2.7: Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

[0023] Furthermore, the preprocessing in step S1 includes adjusting contrast, balancing grayscale, normalizing size, and filtering.

[0024] Noise removal.

[0025] Furthermore, step S2.2 specifically includes:

[0026] Step S2.2.1: Normalize the preprocessed image to obtain the normalized image;

[0027] Step S2.2.2: Perform a two-dimensional convolution operation on the normalized image, add the result of the operation to the preprocessed image, and obtain feature vectors of multiple dimensions.

[0028] Furthermore, step S2.3 specifically includes:

[0029] Step S2.3.1: Normalize the feature vectors of multiple dimensions; obtain the normalized feature vectors;

[0030] Step S2.3.2: Perform point-to-point convolution and depthwise convolution on the normalized feature vector to obtain the aggregated first feature vector.

[0031] Furthermore, the global-local sparse attention module in step S2.4 includes a multi-head attention with mixed scales. The aggregated first feature vector is input into the multi-head attention with mixed scales to obtain multiple local saliency maps and feature vectors containing global and local information. The multiple local saliency maps are summed to obtain a feature saliency map.

[0032] Furthermore, the dual-layer perceptron output module described in step S2.7 consists of a first linear layer, a second linear layer, an activation function layer, and a normalization layer, wherein the normalization layer, the first linear layer, the activation function layer, and the second linear layer are connected in sequence.

[0033] Further, step S2.7 includes: the final feature vector is first processed by a normalization layer, then the normalized feature vector is input into the first linear layer for fusion to obtain a fused feature vector, the fused feature vector is input into the activation function layer for nonlinear transformation, and the transformed feature vector is input into the second linear layer to obtain a classification result.

[0034] To achieve another objective of the present invention, the present invention provides a workpiece defect classification system based on a convolutional-Transformer fusion network, comprising:

[0035] Preprocessing module: used to acquire images of surface defects on the workpiece to be classified and to preprocess the images;

[0036] Classification module: Used to build a fusion network. The preprocessed image is input into the fusion network to obtain the defect classification result, specifically including:

[0037] A fusion network is constructed, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction module includes a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module.

[0038] The preprocessed image is input into the convolutional coding module to obtain feature vectors of multiple dimensions. The convolutional coding module calculates the feature vectors using the following formula:

[0039]

[0040] Where BN represents a normalization layer, and Conv represents a two-dimensional convolution computation. For input data, To output the feature vector;

[0041] The feature vector is input into the local aggregation module to obtain the aggregated first feature vector;

[0042] The aggregated first feature vector is input into the global-local sparse attention module to obtain a feature saliency map;

[0043] The saliency map is input into the guided aggregation module to obtain the aggregated second feature vector, specifically including:

[0044] Step S2.5.1: Divide the feature saliency map into n sub-regions on average;

[0045] Step S2.5.2: Divide the patch in the input feature vector into n feature vectors according to the n sub-regions;

[0046] Step S2.5.3: Set the corresponding aggregation rate based on the salience of the sub-region;

[0047] Step S2.5.4: Aggregate the n feature vectors using the aggregation rate to obtain multiple sets of aggregated feature vectors;

[0048] Step S2.5.5: Concatenate the aggregated feature vectors to obtain the aggregated second feature vector;

[0049] The aggregated second feature vector is input into the downsampling layer to obtain the sampled feature vector;

[0050] Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

[0051] Compared with the prior art, the advantages of this invention are as follows:

[0052] This invention replaces the Patch Embedding layer in ViT with a convolutional coding module to perform preliminary sampling on the input working surface defect image data, mining shallow image features and preventing ViT from directly segmenting the image and destroying some features. This improves the network's ability to model shallow image features, thereby increasing the accuracy of defect classification. Furthermore, it introduces a guided aggregation module in the salient regions of the image to retain more tokens in fine-grained information and fewer image patches in secondary regions, maintaining a self-focused global view and reducing computational load. In addition, multiple downsampling modules are added to continuously shrink the size of the feature map without reducing the network's computational load. Attached Figure Description

[0053] Figure 1 This is a flowchart of a workpiece defect classification method based on a convolutional-Transformer fusion network according to an embodiment of the present invention;

[0054] Figure 2 This is a block diagram of a workpiece defect classification system based on a convolutional-transformer fusion network according to an embodiment of the present invention;

[0055] Figure 3 This is a flowchart of step S2 of a workpiece defect classification method based on a convolutional-Transformer fusion network according to an embodiment of the present invention;

[0056] Figure 4 This is a network structure diagram of a workpiece defect classification method based on a convolutional-transformer fusion network according to an embodiment of the present invention;

[0057] Figure 5 This is a structural diagram of the convolutional coding module of a workpiece defect classification method based on a convolutional-Transformer fusion network according to an embodiment of the present invention;

[0058] Figure 6 This is a diagram of the global-local sparse attention module structure of a workpiece defect classification method based on a convolutional-transformer fusion network according to an embodiment of the present invention. Detailed Implementation

[0059] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0060] Example 1

[0061] like Figure 1 and 3 As shown, a preferred embodiment of the present invention provides a workpiece defect classification method based on a convolutional-Transformer fusion network, comprising:

[0062] Step S1: Obtain images of the workpiece surface defects to be classified and preprocess the images;

[0063] Step S2: Construct a fusion network. Input the preprocessed image into the fusion network to obtain the defect classification results, specifically including:

[0064] Step S2.1: Construct a fusion network. The fusion network includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction modules include a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module.

[0065] Step S2.2: Input the preprocessed image into the convolutional coding module to obtain feature vectors of multiple dimensions;

[0066] Step S2.3: Input the feature vector into the local aggregation module to obtain the first feature vector after aggregation;

[0067] Step S2.4: Input the aggregated first feature vector into the global-local sparse attention module to obtain the feature saliency map;

[0068] Step S2.5: Input the saliency map into the guided aggregation module to obtain the aggregated second feature vector;

[0069] Step S2.6: Input the aggregated second feature vector into the downsampling layer to obtain the sampled feature vector;

[0070] Step S2.7: Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

[0071] This embodiment replaces the Patch Embedding layer in ViT with a convolutional coding module to perform preliminary sampling on the input working surface defect image data, mining shallow image features and preventing ViT from directly segmenting the image and destroying some features. This improves the network's ability to model shallow image features, thereby improving the accuracy of defect classification. Furthermore, by introducing a guided aggregation module in the salient region of the image, more tokens are retained in fine-grained information, while fewer image patches are retained in secondary regions to maintain a self-focused global view, thereby reducing the computational load. In addition, multiple downsampling modules are added to continuously shrink the size of the feature map without reducing the network's computational load.

[0072] Example 2

[0073] like Figure 1 and 3 As shown, a preferred embodiment of the present invention provides a workpiece defect classification method based on a convolutional-Transformer fusion network, comprising:

[0074] Step S1: Obtain images of the workpiece surface defects to be classified and preprocess the images;

[0075] In this embodiment, the preprocessing in step S1 includes adjusting contrast, balancing grayscale, standardizing size, and filtering noise.

[0076] Step S2: Construct a fusion network. Input the preprocessed image into the fusion network to obtain the defect classification results, specifically including:

[0077] Step S2.1: Construct a fusion network, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is followed by a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction modules include a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module; the specific structure diagram is shown below. Figure 4 and 5 As shown;

[0078] Step S2.2: Input the preprocessed image into the convolutional coding module to obtain feature vectors of multiple dimensions;

[0079] In this embodiment, the structure diagram of the convolutional coding module is as follows: Figure 5 As shown, step S2.2 specifically includes:

[0080] Step S2.2.1: Normalize the preprocessed image to obtain the normalized image;

[0081] Step S2.2.2: Perform a two-dimensional convolution operation on the normalized image, and add the result to the preprocessed image to obtain a multi-dimensional feature vector. The convolutional coding module consists of convolutional layers, normalization layers, and activation functions. It performs preliminary sampling of local features and low-level semantic information in the input image through convolution operations, increasing the dimension of the feature map. Simultaneously, it reduces the size of the feature map by controlling the convolution stride. Finally, a 1*1 convolution is used to strengthen the information fusion between channels, and the number of channels is adjusted to meet the input requirements of the subsequent local aggregation module. Residual connections are also added to enhance information flow and improve model performance. The calculation process of the convolutional coding module is as follows:

[0082]

[0083] In the formula Input data (first layer is image data) The following is the feature vector. ... ), The output feature vector is generated by BN, which is a normalization layer, and Conv is a two-dimensional convolutional layer.

[0084] Step S2.3: Input the feature vector into the local aggregation module to obtain the first feature vector after aggregation;

[0085] In this embodiment, step S2.3 specifically includes:

[0086] Step S2.3.1: Normalize the feature vectors of multiple dimensions; obtain the normalized feature vectors;

[0087] Step S2.3.2: Perform point-to-point convolution and depthwise convolution on the normalized feature vector to obtain the aggregated first feature vector. Using point-to-point and depthwise convolution, the input feature vectors are aggregated from adjacent patches into one, shortening the input feature sequence while preserving feature information and reducing spatial redundancy in the image. For each patch, information is gathered into a local window of size k*k, using point-to-point convolution (1*1) and depthwise convolution (3*3). The input data is first normalized, then subjected to point-to-point and depthwise convolution operations. The calculation flow of the local aggregation module is determined by the following formula:

[0088]

[0089] Where PW represents point-wise convolution and DW represents depthwise convolution. Since the number of feature maps generated after depthwise separable convolution is the same as the number of channels, it is impossible to expand the feature maps. Therefore, point-wise convolution is needed to combine these feature maps and generate new feature maps. The specific generation is determined by the following formula:

[0090]

[0091] Step S2.4: Input the aggregated first feature vector into the global-local sparse attention module to obtain the feature saliency map;

[0092] In this embodiment, the global-local sparse attention module in step S2.4 mainly consists of a multi-head attention model with mixed scales. The aggregated first feature vector is input into the multi-head attention model with mixed scales to obtain multiple local saliency maps and feature vectors containing global and local information. The multiple local saliency maps are summed to obtain a feature saliency map. The output feature X of the local aggregation module is projected onto the Query(Q), Key(K), and Value(V) values. The multi-head self-attention uses H independent heads. Typically, these H independent heads perform in the same local region, thus lacking head diversity. Therefore, the H heads are evenly inserted into H groups, and mixed-scale and multi-receptive field attention are injected into these H groups, where each group has H heads, and self-attention calculations are performed separately, as shown in the appendix. Figure 3 In the i-th head belonging to the j-th group, with a scale of sj (where j = 1, ..., h), every sj*sj patches in {K,V} are merged into one patch. Next, {Q,K,V} is divided into windows. The window size for {K,V} is set to M and remains constant across all groups. To align the window sizes of {Q} and {K,V} with the merged patches in {K,V}, the window size for {Q} is chosen to be sjM*sjM, which is sj times larger than that of {K,V}, significantly expanding the receptive field of attention by sj times. The attention mechanism is determined by the following equation:

[0093]

[0094] The local saliency plot is determined by the following formula:

[0095]

[0096] In the formula, P(X,sjM) represents a window partition with a size of sjM*sjM. All patches with the {K,V} key value in the attention mechanism involve the {Q} key value, achieving global information extraction.

[0097] Then, sum the multiple local saliency maps to obtain the eigenvalue map:

[0098]

[0099] Where S is a pair of S i The final saliency plot of the summation is used at a mixed scale with both global and fine-grained information to guide the aggregation module.

[0100] Step S2.5: Input the saliency map into the guided aggregation module to obtain the aggregated second feature vector;

[0101] In this embodiment, step S2.5 specifically includes:

[0102] Step S2.5.1: Divide the feature saliency map into n sub-regions on average;

[0103] Step S2.5.2: Divide the patch in the input feature vector into n feature vectors according to the n sub-regions;

[0104] Step S2.5.3: Set the corresponding aggregation rate based on the salience of the sub-region;

[0105] Step S2.5.4: Aggregate the n feature vectors using the aggregation rate to obtain multiple sets of aggregated feature vectors;

[0106] Step S2.5.5: Concatenate the aggregated feature vectors to obtain the aggregated second feature vector.

[0107] This embodiment first projects the input feature map X∈Rh*w*c onto Query(Q), Key(K), and Value(V). To reduce computational cost while maintaining the size of the post-focused feature map, the Guided Aggregation Module (GAM) aggregates the patches for K and V, and fixes the length of Q. The saliency map S∈Rh*w contains regional information of multi-granularity saliency. The values ​​of the saliency map are sorted in ascending order, and S is divided into n sub-regions S1, ..., Sn. Therefore, Sn and S1 are the most important and least important regions, respectively. Simultaneously, based on S1, ..., Sn, all patches in X are divided into X1, ..., Xn. Different aggregation rates r1, ..., rn are set for regions of different saliencies, so that each sub-region has an aggregation rate; the more important the sub-region, the smaller the aggregation rate. Therefore, the GAM aggregates the grouped input features X1, ..., Xn and redistributes patches to each group with a different aggregation rate. The aggregation function is determined by the following formula:

[0108]

[0109] r represents the aggregation rate at which r patches are aggregated into one patch. F(X,r) is the aggregation function, implemented through a fully connected layer with input dimension r and output dimension 1.

[0110] The second feature vector after aggregation is determined by the following formula:

[0111]

[0112] Step S2.6: Input the aggregated second feature vector into the downsampling layer to obtain the sampled feature vector;

[0113] Step S2.7: Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

[0114] In this embodiment, the two-layer perceptron (FFN) output module in step S2.7 consists of a first linear layer, a second linear layer, an activation function layer, and a normalization layer, wherein the normalization layer, the first linear layer, the activation function layer, and the second linear layer are connected sequentially. The two-layer perceptron (FFN) output module is determined by the following formula:

[0115]

[0116] Example 3

[0117] like Figure 2 As shown, a preferred embodiment of the present invention provides a workpiece defect classification system based on a convolutional-transformer fusion network, comprising:

[0118] Preprocessing module: used to acquire images of surface defects on the workpiece to be classified and to preprocess the images;

[0119] Classification module: Used to build a fusion network. The preprocessed image is input into the fusion network to obtain the defect classification result, specifically including:

[0120] A fusion network is constructed, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction module includes a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module.

[0121] The preprocessed image is input into the convolutional coding module to obtain feature vectors of multiple dimensions;

[0122] The feature vector is input into the local aggregation module to obtain the aggregated first feature vector;

[0123] The aggregated first feature vector is input into the global-local sparse attention module to obtain a feature saliency map;

[0124] The saliency map is input into the guided aggregation module to obtain the aggregated second feature vector;

[0125] The aggregated second feature vector is input into the downsampling layer to obtain the sampled feature vector;

[0126] Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

[0127] This embodiment replaces the Patch Embedding layer in ViT with a convolutional coding module to perform preliminary sampling on the input working surface defect image data, mining shallow image features and preventing ViT from directly segmenting the image and destroying some features. This improves the network's ability to model shallow image features, thereby improving the accuracy of defect classification. Furthermore, by introducing a guided aggregation module in the salient region of the image, more tokens are retained in fine-grained information, while fewer image patches are retained in secondary regions to maintain a self-focused global view, thereby reducing the computational load. In addition, multiple downsampling modules are added to continuously shrink the size of the feature map without reducing the network's computational load.

[0128] In summary, this invention provides a workpiece defect classification method and system based on a convolutional-transformer fusion network. It replaces the patch embedding layer in ViT with a convolutional coding module to perform preliminary sampling of the input work surface defect image data, mining shallow image features and preventing ViT from directly segmenting the image and destroying some features. This improves the network's ability to model shallow image features, thereby increasing the accuracy of defect classification. Furthermore, it introduces a guided aggregation module in salient regions of the image, retaining more tokens in fine-grained information and fewer image patches in secondary regions to maintain a self-focused global view, thus reducing computational load. Additionally, it adds multiple downsampling modules to continuously shrink the feature map size without reducing network computation.

[0129] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for classifying workpiece defects based on a convolutional-Transformer fusion network, characterized in that, include: Step S1: Obtain images of surface defects on the workpiece to be classified and preprocess the images; Step S2: Construct a fusion network by inputting the preprocessed image into the fusion network to obtain defect classification results, specifically including: Step S2.1: Construct a fusion network, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction module includes a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module. Step S2.2: Input the preprocessed image into the convolutional coding module to obtain feature vectors of multiple dimensions. The convolutional coding module calculates the feature vectors using the following formula: Where BN represents a normalization layer, and Conv represents a two-dimensional convolution computation. For input data, To output the feature vector; Step S2.3: Input the feature vector into the local aggregation module to obtain the first feature vector after aggregation; Step S2.4: Input the aggregated first feature vector into the global-local sparse attention module to obtain the feature saliency map; Step S2.5: Input the saliency map into the guided aggregation module to obtain the aggregated second feature vector, specifically including: Step S2.5.1: Divide the feature saliency map into n sub-regions on average; Step S2.5.2: Divide the patch in the input feature vector into n feature vectors according to the n sub-regions; Step S2.5.3: Set the corresponding aggregation rate based on the salience of the sub-region; Step S2.5.4: Aggregate the n feature vectors using the aggregation rate to obtain multiple sets of aggregated feature vectors. quantity; Step S2.5.5: Concatenate the aggregated feature vectors to obtain the aggregated second feature vector; Step S2.6: Input the aggregated second feature vector into the downsampling layer to obtain the sampled feature vector; Step S2.7: Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

2. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 1, characterized in that, The preprocessing in step S1 includes adjusting contrast, balancing grayscale, normalizing size, and filtering noise.

3. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 1, characterized in that, Step S2.2 specifically includes: Step S2.2.1: Normalize the preprocessed image to obtain the normalized image; Step S2.2.2: Perform a two-dimensional convolution operation on the normalized image, add the result of the operation to the preprocessed image, and obtain feature vectors of multiple dimensions.

4. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 3, characterized in that, Step S2.3 specifically includes: Step S2.3.1: Normalize the feature vectors of multiple dimensions; obtain the normalized feature vectors; Step S2.3.2: Perform point-to-point convolution and depthwise convolution on the normalized feature vector to obtain the aggregated first feature vector.

5. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 4, characterized in that, The global-local sparse attention module in step S2.4 includes a multi-head attention with mixed scales. The aggregated first feature vector is input into the multi-head attention with mixed scales to obtain multiple local saliency maps and feature vectors containing global and local information. The multiple local saliency maps are summed to obtain a feature saliency map.

6. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 5, characterized in that, The dual-layer perceptron output module described in step S2.7 consists of a first linear layer, a second linear layer, an activation function layer, and a normalization layer, wherein the normalization layer, the first linear layer, the activation function layer, and the second linear layer are connected in sequence.

7. The workpiece defect classification method based on a convolutional-Transformer fusion network according to claim 6, characterized in that, Step S2.7 includes: the final feature vector is first processed by a normalization layer, then the normalized feature vector is input into the first linear layer for fusion to obtain a fused feature vector, the fused feature vector is input into the activation function layer for nonlinear transformation, and the transformed feature vector is input into the second linear layer to obtain a classification result.

8. A workpiece defect classification system based on a convolutional-transformer fusion network, characterized in that, include: Preprocessing module: used to acquire images of surface defects on the workpiece to be classified and to preprocess the images; Classification module: Used to build a fusion network. The preprocessed image is input into the fusion network to obtain the defect classification result, specifically including: A fusion network is constructed, which includes at least four feature extraction modules, at least four downsampling layers, and a two-layer perceptron output module. Each feature extraction module is connected to a downsampling layer, and the last downsampling layer is connected to the two-layer perceptron output module. The feature extraction module includes a convolutional coding module, a local aggregation module, a global-local sparse attention module, and a guided aggregation module. The preprocessed image is input into the convolutional coding module to obtain feature vectors of multiple dimensions. The convolutional coding module calculates the feature vectors using the following formula: Where BN represents a normalization layer, and Conv represents a two-dimensional convolution computation. For input data, To output the feature vector; The feature vector is input into the local aggregation module to obtain the aggregated first feature vector; The aggregated first feature vector is input into the global-local sparse attention module to obtain a feature saliency map; The saliency map is input into the guided aggregation module to obtain the aggregated second feature vector, specifically including: Step S2.5.1: Divide the feature saliency map into n sub-regions on average; Step S2.5.2: Divide the patch in the input feature vector into n feature vectors according to the n sub-regions; Step S2.5.3: Set the corresponding aggregation rate based on the salience of the sub-region; Step S2.5.4: Aggregate the n feature vectors using the aggregation rate to obtain multiple sets of aggregated feature vectors. quantity; Step S2.5.5: Concatenate the aggregated feature vectors to obtain the aggregated second feature vector; The aggregated second feature vector is input into the downsampling layer to obtain the sampled feature vector; Repeat steps S2.2 to S2.6 at least four times to obtain the final feature vector. Input the final feature vector into the output module of the two-layer perceptron to obtain the classification result.

Images