Multi-class welding spot defect classification method, system and equipment based on Transform and channel interaction and medium

Abstract

The invention relates to a multi-category welding spot defect classification method, system and equipment based on Transform and channel interaction and a medium. The method comprises the following steps: preprocessing an input image, and respectively carrying out division, embedding, position coding and category information embedding on an input support image and an input query image; an image block embedding sequence containing position and category information is transmitted to a Transform encoder to extract embedding features containing category related information and original image blocks; performing adaptive adjustment on the image block embedding features of each channel by using the category-related embedding vector and the global visual context vector; judging whether the support images and the query images belong to the same category or defect type or not by calculating the similarity of the support images and the query images; outputting defect categories of the images through a classification network; according to the method, the precision of welding spot defect classification is effectively improved, and the problems of category confusion and small sample learning confronted by a traditional method are solved especially in the scene of various defect types and small sample size.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and in particular to a multi-category solder joint defect classification method, system, device and medium based on Transformer and channel interaction. Background Art

[0002] Solder spot defect detection has important applications in modern manufacturing, especially in high-precision fields such as electronics, automobiles, and aviation. With the complexity of welding processes, there are many types of solder spot defects and they vary widely. Traditional detection methods are difficult to meet the needs of efficiency and accuracy. Especially in defect classification, the characteristics of solder spot defects are highly similar and the sample size is small. Deep learning models face the problems of data scarcity and category confusion.

[0003] In recent years, deep learning-based solder joint defect detection has made some progress, but it still faces two major challenges: first, the defect types are complex and local features are prone to cause category confusion; second, under small sample learning conditions, the classification accuracy is limited. To solve these problems, the Transformer model has gradually been applied to visual tasks with its powerful global information modeling capabilities and channel interaction mechanism. This paper proposes a multi-category solder joint defect classification method based on Transformer and channel interaction, which improves the accuracy of solder joint defect classification through global context information modeling and fine-grained similarity measurement. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a multi-category solder joint defect classification method based on Transformer and channel interaction. This method effectively improves the accuracy of solder joint defect classification by introducing global visual context information and feature channel interaction mechanism, especially in scenarios with diverse defect types and small sample size, overcoming the category confusion and small sample learning problems faced by traditional methods. At the same time, by constructing a fine-grained similarity measurement matrix, the model's ability to distinguish between different defect categories is further enhanced, which has strong robustness and broad application prospects.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a multi-category solder joint defect classification method based on Transformer and channel interaction, comprising the following steps: S1. Input image preprocessing: divide, embed, position encode and embed category information of input support image and query image respectively, and generate image block embedding sequence containing position and category information. ; S2: Construct a category embedding extraction module to embed image blocks containing position and category information into a sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. , enhance global context awareness; S3, build feature channel interaction module, use category-related embedding vector With the global visual context vector Embed features for each channel image patch Make adaptive adjustments; S4, construct a similarity measurement module to determine whether the support image and the query image belong to the same category or defect type by calculating the similarity between them; S5. Classification output: Based on feature fusion and similarity score, the defect category of the image is output through the classification network.

[0006] Furthermore, in step S1, image preprocessing specifically includes: dividing the image blocks, the input image is divided into a plurality of small, non-overlapping image blocks, each image block contains a local area of ​​the image.

[0007] Furthermore, in step S1, an image block embedding sequence containing position and category information is generated. The specific process includes the following steps: S11, image block embedding vector mapping, each image block is mapped to the feature space through linear projection to obtain the image block embedding vector ; S12, then embed the vector into the image block Processing to obtain the initial image block embedding vector sequence : ; in, Indicates the number of features in the sequence; S13, category information embedding, the global category information of the image is embedded through a learnable category embedding vector with global category information To represent, placed in the initial image block embedding vector sequence The starting position of the image block helps the model to combine the global category information when processing the image block and obtain the image block embedding sequence after the category embedding processing. : ; S14, position encoding, because the Transformer architecture itself does not have spatial position information, so each image block is embedded into a sequence after category embedding Adding a position encoding matrix , represents the position of each image block and is initialized as a set of learnable parameters to obtain an image block embedding sequence containing position and category information : .

[0008] Further, in step S2, the image block containing the position and category information is embedded into the sequence The Transformer encoder is used to extract the embedded features containing category-related information and the original image blocks. The specific process includes the following steps: S21, the image block embedding sequence containing position and category information is sent to the layer normalization of the Transformer stage, which consists of Transformer encoders are repeatedly stacked. layer, In the Transformer encoder of , layer normalization is used to normalize the input image patch embedding sequence; S22, through multi-head self-attention and MLP processing, and then in the multi-head self-attention layer, use multiple different attention heads to calculate the sequence The attention output of , the attention output and the normalized output are connected together using a residual connection: ; in, is the output of the previous encoder, is the output of the current encoder layer after being processed by the multi-head self-attention layer; S23, yes After layer normalization again, it is sent to the MLP layer, which contains two fully connected layers and a GELU activation function: ; in, Represents the output of the encoder at this layer; S24, category information aggregation and output, to obtain the final output of the Lth layer: ; in, It is the final output of the Transformer encoder stack, which contains category-related information and features of each image block of the original image; It is a category-related embedding vector, which represents the high-level semantic information of the entire image and accumulates information from all blocks of the image; It is the image block embedding sequence after feature extraction.

[0009] Further, in step S3, the category-related embedding vector With the global visual context vector Embed features for each channel image patch Perform adaptive adjustment. The specific process includes the following steps: S31, calculate the global visual context vector, and , embed each image block into features Average to get the global visual context vector : ; S32, generate category guidance vector , embedding the category-related vector With the global visual context vector Concatenate and concatenate to form a new vector , input the concatenated vector into a two-layer MLP to generate a category guidance vector: ; in, represents the sigmoid activation function, and represents the weights of the two-layer MLP, and represents the bias of the two-layer MLP; S33, adjust the image block features, and finally guide the category vector Add to each image block embedding feature The channel of , thereby guiding the characteristics of each channel, and the final sequence is: .

[0010] Furthermore, in step S4, the specific process includes the following steps: S41, Mahalanobis distance calculation, the first i Image patch embedding sequence and the query image set j Image patch embedding sequence Input into the similarity measurement module and calculate the Mahalanobis distance between the two: ; in, Represents the similarity matrix i Line j Elements of a column; Indicates the first iThe average value of all feature embeddings in the image patch embedding sequence; Indicates i The covariance of all feature embeddings in the image patch embedding sequence; S42, similarity score and final output, after calculating the dense matrix S After that, it is flattened into a one-dimensional vector so that each element corresponds to the similarity of a support image and query image embedding pair. Subsequently, the flattened vector is input into MLP and aggregated into a similarity score through multiple layers of nonlinear transformations. This score represents the overall similarity between the support image and the query image.

[0011] Further, in step S41, the first i The average of all feature embeddings in the image patch embedding sequence and i The covariance of all feature embeddings in the image patch embedding sequence , and their calculation methods are: ; ; in, Indicates the number of features in the sequence; Indicates the first Features.

[0012] By means of the above technical solution, the present invention provides a multi-category solder joint defect classification method based on Transformer and channel interaction, which has at least the following beneficial effects: The present invention realizes a multi-category solder joint defect classification method based on Transformer and channel interaction. Firstly, by constructing a category embedding extraction module, a stacked Transformer structure is used to extract embedded features containing category-related information and original image blocks, which can integrate the content and category relevance of the entire image and help the model better mine the global and local information of the image; secondly, by introducing global visual context information and feature channel interaction mechanism, the category-related embedding vector and the global visual context vector are used to embed features of the image blocks of each channel. Adaptive adjustment is performed to effectively improve the accuracy of solder joint defect classification, especially in scenarios with diverse defect types and small sample size, overcoming the category confusion and small sample learning problems faced by traditional methods; at the same time, by constructing a fine-grained similarity matrix, the similarity between support image and query image pairs is quantified, making the classification of features with strong consistency more accurate, thereby further enhancing the model's ability to distinguish between different defect categories, with strong robustness and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a flowchart of the main links of the multi-category solder joint defect classification method based on Transformer and channel interaction proposed by the present invention; Figure 2 It is an algorithm flow chart of a multi-category solder joint defect classification method based on Transformer and channel interaction proposed by the present invention; Figure 3 This is a schematic diagram of the principle of the Transformer encoder proposed in the present invention. DETAILED DESCRIPTION

[0014] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0015] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0016] Please refer to Figure 1-Figure 3 , shows a specific implementation of this embodiment. This embodiment of the present invention implements a multi-category solder joint defect classification method based on Transformer and channel interaction. First, by constructing a category embedding extraction module, a stacked Transformer structure is used to extract embedded features containing category-related information and original image blocks. , which can integrate the content and category relevance of the entire image, helping the model to better mine the global and local information of the image; secondly, by introducing global visual context information and feature channel interaction mechanism, using category-related embedding vectors With the global visual context vector Embed features for each channel image patch Adaptive adjustment is performed to effectively improve the accuracy of solder joint defect classification, especially in scenarios with diverse defect types and small sample size, overcoming the category confusion and small sample learning problems faced by traditional methods; at the same time, by constructing a fine-grained similarity matrix, the similarity between support image and query image pairs is quantified, making the classification of features with strong consistency more accurate, thereby further enhancing the model's ability to distinguish between different defect categories, with strong robustness and broad application prospects.

[0017] Please refer to Figure 1 , this embodiment proposes a multi-category solder joint defect classification method based on Transformer and channel interaction, the method comprising the following steps: S1. Input image preprocessing: divide, embed, position encode and embed category information of input support image and query image respectively, and generate image block embedding sequence containing position and category information. ; As a preferred implementation of step S1, in step S1, image preprocessing specifically includes: dividing the image blocks, the input image is divided into multiple small, non-overlapping image blocks, each image block contains a local area of ​​the image; this division method can not only improve the processing efficiency of local features, but also maintain the spatial structure information of the image, providing valuable local features for subsequent steps.

[0018] In addition, in step S1, an image block embedding sequence containing position and category information is generated The specific process includes the following steps: S11, image block embedding vector mapping, each image block is mapped to the feature space through linear projection to obtain the image block embedding vector ,This process converts the image block into a vector in a high-dimensional space, which can effectively represent the local features of the image block; S12, then embed the vector into the image block Processing to obtain the initial image block embedding vector sequence : ; in, Indicates the number of features in the sequence; S13, category information embedding, the global category information of the image is embedded through a learnable category embedding vector with global category information To represent, placed in the initial image block embedding vector sequence The starting position of the image block helps the model to combine the global category information when processing the image block and obtain the image block embedding sequence after the category embedding processing. : ; S14, position encoding, because the Transformer architecture itself does not have spatial position information, so each image block is embedded into a sequence after category embedding Adding a position encoding matrix , represents the position of each image block and is initialized as a set of learnable parameters to obtain an image block embedding sequence containing position and category information : ; In this embodiment, by adding the position encoding to the embedding of each image block, the spatial position information of the image can be retained, thereby integrating the content and category relevance of the entire image and helping the model to better mine the global and local information of the image; in this way, each image block embedding contains its position information so that the spatial relationship can be retained in the self-attention mechanism.

[0019] S2: Construct a category embedding extraction module to embed image blocks containing position and category information into a sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. , enhance global context awareness; As a preferred implementation of step S2, in step S2, the image block containing the position and category information is embedded into the sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. The specific process includes the following steps: S21, the image block embedding sequence containing position and category information is sent to the layer normalization of the Transformer stage, which consists of Transformer encoders are repeatedly stacked, and the principle is as follows Figure 3 As shown. layer, In the Transformer encoder of , layer normalization is used to normalize the input image patch embedding sequence; S22, through multi-head self-attention and MLP processing, and then in the multi-head self-attention layer, use multiple different attention heads to calculate the sequence The attention output of , the attention output and the normalized output are connected together using a residual connection: ; in, is the output of the previous encoder, is the output of the current encoder layer after being processed by the multi-head self-attention layer; S23, yes After layer normalization again, it is sent to the MLP layer, which contains two fully connected layers and a GELU activation function: ; in, Represents the output of the encoder at this layer; S24, category information aggregation and output, to obtain the final output of the Lth layer: ; in, It is the final output of the Transformer encoder stack, which contains category-related information and features of each image block of the original image; It is a category-related embedding vector, which represents the high-level semantic information of the entire image and accumulates information from all blocks of the image; It is the image block embedding sequence after feature extraction.

[0020] In this embodiment, if Figure 2 As shown in Figure 2, category embedding extraction uses a stacked Transformer structure to extract embedded features containing category-related information and the original image blocks. The purpose of this step is to extract the category-related information of the image through the Transformer encoder and effectively integrate it into the feature representation of the image block, which can integrate the content and category relevance of the entire image and help the model better mine the global and local information of the image.

[0021] S3, build feature channel interaction module, use category-related embedding vector With the global visual context vector Embed features for each channel image patch Make adaptive adjustments; As a preferred implementation of step S3, in step S3, using the category-related embedding vector With the global visual context vector Embed features for each channel image patch Perform adaptive adjustment. The specific process includes the following steps: S31, calculate the global visual context vector, and , embed each image block into features Average to get the global visual context vector : ; S32, generate category guidance vector , embedding the category-related vector With the global visual context vector Concatenate and concatenate to form a new vector , input the concatenated vector into a two-layer MLP to generate a category guidance vector: ; in, represents the sigmoid activation function, and represents the weights of the two-layer MLP, and represents the bias of the two-layer MLP; S33, adjust the image block features, and finally guide the category vector Add to each image block embedding feature The channel of , thereby guiding the characteristics of each channel, and the final sequence is: .

[0022] In this embodiment, if Figure 2 As shown in the figure, feature channel interaction, by introducing global visual context information and feature channel interaction mechanism, using category-related embedding vector With the global visual context vector Embed features for each channel image patch Adaptive adjustment is performed by applying the global visual context vector on different channels of feature extraction, so that the model can more selectively strengthen the visual features related to the category; thereby adjusting the features of each channel, and the category guidance vector It can adaptively adjust the image block embedding features according to the fine-grained features of different categories, enhance category-specific visual features, and thus improve the performance in small-sample multi-category tasks; it effectively improves the accuracy of solder joint defect classification, especially in scenarios with diverse defect types and small sample size, overcoming the category confusion and small-sample learning problems faced by traditional methods.

[0023] S4, construct a similarity measurement module to determine whether the support image and the query image belong to the same category or defect type by calculating the similarity between them; As a preferred implementation of step S4, in step S4, the specific process includes the following steps: S41, Mahalanobis distance calculation, the first i Image patch embedding sequence and the query image set j Image patch embedding sequence Input into the similarity measurement module and calculate the Mahalanobis distance between the two: ; in, Represents the similarity matrix i Line j Elements of a column; Indicates the first i The average value of all feature embeddings in the image patch embedding sequence; Indicates i The covariance of all feature embeddings in the image block embedding sequence is calculated as: ; ; in, Indicates the number of features in the sequence; Indicates the first Features S42, similarity score and final output, such as Figure 2 As shown, when calculating the dense matrix S After that, it is flattened into a one-dimensional vector so that each element corresponds to the similarity of a support image and query image embedding pair. Subsequently, the flattened vector is input into MLP and aggregated into a similarity score through multiple layers of nonlinear transformations. This score represents the overall similarity between the support image and the query image.

[0024] In this embodiment, in the task of classifying solder joint defects with strong inter-class similarity, the Mahalanobis distance can more accurately measure the differences between different categories, especially when the sample distributions between categories are relatively similar, the Mahalanobis distance can help avoid errors caused by feature irrelevance; at the same time, by constructing a fine-grained similarity matrix, the similarity between the support image and the query image pair is quantified, making the classification of features with strong consistency more accurate, thereby further enhancing the model's ability to distinguish between different defect categories, and having strong robustness and broad application prospects.

[0025] S5. Classification output. Based on feature fusion and similarity score, the obtained similarity score is used as the output of the classification task, and the defect category of the image is output through the classification network.

[0026] Specifically, the present application also provides a multi-category solder joint defect classification system based on Transformer and channel interaction, including: An image preprocessing module, wherein the image preprocessing module is used to detect the target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification; A data set generation module, which is used to divide, embed, position encode and embed category information of the input support image and query image respectively, and generate an image block embedding sequence containing position and category information ; A category embedding extraction module is used to embed image blocks containing position and category information into a sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. , enhance global context awareness; A feature channel interaction module is used to utilize category-related embedding vectors With the global visual context vector Embed features for each channel image patch Make adaptive adjustments; A similarity measurement module, wherein the similarity measurement module is used to determine whether the support image and the query image belong to the same category or defect type by calculating the similarity between them; The classification output module is used to output the defect category of the image through a classification network according to feature fusion and similarity score.

[0027] Specifically, the present application also provides an electronic device, including: a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned multi-category solder joint defect classification method based on Transformer and channel interaction.

[0028] Specifically, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned multi-category solder joint defect classification method based on Transformer and channel interaction.

[0029] Thus, a multi-category solder joint defect classification method based on Transformer and channel interaction is realized, aiming to improve the accuracy of solder joint defect fine classification and solve the two main challenges in the task of solder joint defect classification: one is the diversity of solder joint defect types and the other is the consistency of solder joint defect features. In view of the complex types of solder joint defects and the small sample size, the channel information interaction mechanism is adopted to introduce the global visual context vector to strengthen the information interaction between feature channels, so as to effectively capture a wide range of context information and avoid confusion in the classification process of multi-class defect features. At the same time, a fine-grained similarity measurement matrix is ​​constructed to maximize the use of image features under few-sample conditions by calculating the similarity between image blocks, thereby improving the classification accuracy. The proposed algorithm consists of three main modules: (1) a category embedding extraction module that uses stacked Transformer blocks to capture the global features of the image; (2) a feature channel interaction module that promotes the joint representation of different feature channels; and (3) a similarity measurement matrix that supports the similarity between the image and the query image through the Mahalanobis distance calculation for accurate classification. This method has significant advantages in the accurate classification of solder joint defects and small sample learning scenarios.

[0030] In summary, the present invention can realize multi-category solder joint defect classification based on Transformer and channel interaction, and can effectively deal with the generalization problem in the case of few samples.

[0031] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0032] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0033] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A multi-category solder joint defect classification method based on Transformer and channel interaction, characterized in that: The following steps are involved: S1. Input image preprocessing: divide, embed, position encode and embed category information of input support image and query image respectively, and generate image block embedding sequence containing position and category information. ; S2: Construct a category embedding extraction module to embed image blocks containing position and category information into a sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. , enhance global context awareness; S3, build feature channel interaction module, use category-related embedding vector With the global visual context vector Embed features for each channel image patch Make adaptive adjustments; S4, construct a similarity measurement module to determine whether the support image and the query image belong to the same category or defect type by calculating the similarity between them; S5. Classification output: Based on feature fusion and similarity score, the defect category of the image is output through the classification network.

2. According to claim 1, a multi-category solder joint defect classification method based on Transformer and channel interaction is characterized in that: In step S1, image preprocessing specifically includes: dividing the image blocks, the input image is divided into a plurality of small, non-overlapping image blocks, each image block contains a local area of ​​the image.

3. According to claim 1, a multi-category solder joint defect classification method based on Transformer and channel interaction is characterized in that: In step S1, an image block embedding sequence containing position and category information is generated The specific process includes the following steps: S11, image block embedding vector mapping, each image block is mapped to the feature space through linear projection to obtain the image block embedding vector ; S12, then embed the vector into the image block Processing to obtain the initial image block embedding vector sequence : ； in, Indicates the number of features in the sequence; S13, category information embedding, the global category information of the image is embedded through a learnable category embedding vector with global category information To represent, placed in the initial image block embedding vector sequence The starting position of the image block helps the model to combine the global category information when processing the image block and obtain the image block embedding sequence after the category embedding processing. : ； S14, position encoding, since the Transformer architecture itself does not have spatial position information, each image block is embedded into a sequence after category embedding Adding a position encoding matrix , represents the position of each image block and is initialized as a set of learnable parameters to obtain an image block embedding sequence containing position and category information : 。 4. According to claim 1, a multi-category solder joint defect classification method based on Transformer and channel interaction is characterized in that: In step S2, the image blocks containing position and category information are embedded into the sequence The Transformer encoder is used to extract the embedded features containing category-related information and the original image blocks. The specific process includes the following steps: S21, the image block embedding sequence containing position and category information is sent to the layer normalization of the Transformer stage, which consists of Transformer encoders are repeatedly stacked. layer, In the Transformer encoder of , layer normalization is used to normalize the input image patch embedding sequence; S22, through multi-head self-attention and MLP processing, and then in the multi-head self-attention layer, use multiple different attention heads to calculate the sequence The attention output of , the attention output and the normalized output are connected together using a residual connection: ； in, is the output of the previous encoder, is the output of the current encoder layer after being processed by the multi-head self-attention layer; S23, yes After layer normalization again, it is sent to the MLP layer, which contains two fully connected layers and a GELU activation function: ； in, Represents the output of the encoder at this layer; S24, category information aggregation and output, to obtain the final output of the Lth layer: ； in, It is the final output of the Transformer encoder stack, which contains category-related information and features of each image block of the original image; It is a category-related embedding vector, which represents the high-level semantic information of the entire image and accumulates information from all blocks of the image; It is the image block embedding sequence after feature extraction.

5. The multi-category solder joint defect classification method based on Transformer and channel interaction according to claim 4 is characterized in that: In step S3, the category-dependent embedding vector With the global visual context vector Embed features for each channel image patch Perform adaptive adjustment. The specific process includes the following steps: S31, calculate the global visual context vector, and , embed each image block into features Average to get the global visual context vector : ； S32, generate category guidance vector , embedding the category-related vector With the global visual context vector Concatenate and concatenate to form a new vector , the concatenated vector is input into a two-layer MLP to generate a category guidance vector: ； in, represents the sigmoid activation function, and represents the weights of the two-layer MLP, and represents the bias of the two-layer MLP; S33, adjust the image block features, and finally guide the category vector Add to each image block embedding feature The channel of , thereby guiding the characteristics of each channel, and the final sequence is: 。 6. The multi-category solder joint defect classification method based on Transformer and channel interaction according to claim 1 is characterized in that: In step S4, the specific process includes the following steps: S41, Mahalanobis distance calculation, the first i Image patch embedding sequence and the query image set j Image patch embedding sequence Input into the similarity measurement module and calculate the Mahalanobis distance between the two: ； in, Represents the similarity matrix i Line j Elements of a column; Indicates the first i The average value of all feature embeddings in the image patch embedding sequence; Indicates i The covariance of all feature embeddings in the image patch embedding sequence; S42, similarity score and final output, after calculating the dense matrix S After that, it is flattened into a one-dimensional vector so that each element corresponds to the similarity of a support image and query image embedding pair. Subsequently, the flattened vector is input into MLP and aggregated into a similarity score through multiple layers of nonlinear transformations. This score represents the overall similarity between the support image and the query image.

7. The multi-category solder joint defect classification method based on Transformer and channel interaction according to claim 6 is characterized by: In step S41, the first i The average of all feature embeddings in the image patch embedding sequence and i The covariance of all feature embeddings in the image patch embedding sequence , and their calculation methods are: ； ； in, Indicates the number of features in the sequence; Indicates the first Features.

8. A multi-category solder joint defect classification system based on Transformer and channel interaction, characterized in that: include: An image preprocessing module, wherein the image preprocessing module is used to detect the target to be detected by respectively designing a positioning network model and a classification network model to obtain preliminary positioning and classification; A data set generation module, which is used to divide, embed, position encode and embed category information of the input support image and query image respectively, and generate an image block embedding sequence containing position and category information ; A category embedding extraction module is used to embed image blocks containing position and category information into a sequence The Transformer encoder extracts the embedded features containing category-related information and the original image blocks. , enhance global context awareness; A feature channel interaction module is used to utilize category-related embedding vectors With the global visual context vector Embed features for each channel image patch Make adaptive adjustments; A similarity measurement module, wherein the similarity measurement module is used to determine whether the support image and the query image belong to the same category or defect type by calculating the similarity between them; The classification output module is used to output the defect category of the image through a classification network according to feature fusion and similarity score.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-category solder joint defect classification method based on Transformer and channel interaction as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multi-category solder joint defect classification method based on Transformer and channel interaction according to any one of claims 1 to 7 is implemented.

Images