A method and system for identifying surface defects in small sample products based on image recognition

By using a dual-branch feature extraction network and a learnable prototype calibration module, the accuracy and robustness issues of product surface defect identification under small sample conditions are solved, achieving high-precision defect region segmentation and identification.

CN123118592APending Publication Date: 2026-11-17WUHAN RUIKE ZHICHENG INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611457609.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-18
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Under small sample conditions, existing technologies struggle to effectively identify product surface defects, especially due to inaccurate defect segmentation and insufficient robustness caused by differences in multi-scale features and uneven channel-level feature responses.

Method used

A dual-branch feature extraction network is used to decouple features at multiple scales and channels. Combined with a learnable prototype calibration module, a calibration prototype is generated through an attention computation unit. Spatial constraints are applied using a defect region mask to improve the accuracy and robustness of defect region recognition.

Benefits of technology

With very few labeled samples, high-precision localization and classification of product surface defects were achieved, improving the precision of defect area segmentation and adaptability to scale changes, and reducing interference from background information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118592A_ABST
    Figure CN123118592A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for identifying surface defects in products using image recognition in small samples, belonging to the field of product surface defect detection technology. The method includes: acquiring a query image of the product surface to be identified and a support set containing a small number of labeled samples with known defect categories; extracting multi-scale feature maps from the query image and the support set respectively using a dual-branch feature extraction network, and performing channel-level decoupling on the multi-scale feature maps to obtain several feature subspaces; using a learnable prototype calibration module to perform weighted aggregation of the support set features in each feature subspace to obtain a calibration prototype for each defect category; calculating the correlation response map between the feature subspace of the query image and the calibration prototype, and using the correlation response map to segment the defect region to obtain the defect identification result. Through multi-scale feature decoupling and prototype calibration, the identification accuracy and segmentation completeness of product surface defects can be improved under small sample conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of product surface defect detection technology, specifically to a method and system for identifying surface defects in small samples of products based on image recognition. Background Technology

[0002] In the field of product surface defect recognition, deep learning-based methods typically rely on a large number of labeled defect samples for training. However, in actual industrial production, some defect categories occur with extremely low frequency, making it extremely difficult to collect a large number of labeled samples. Therefore, surface defect recognition under small sample conditions has become a pressing technical problem. In existing technologies, when using meta-learning frameworks such as prototype networks for small-sample defect detection, single-scale feature maps extracted by convolutional neural networks are often directly used for prototype calculation and similarity measurement. This approach ignores the differences in the multi-scale representation of product surface defects. Small defects are prone to losing detail in deep features, while larger defects may not be fully perceived due to receptive field limitations. Furthermore, treating all channels as equally important features for prototype calculation fails to distinguish the differentiated visual attributes such as texture, edges, and shape responded by different channel groups, resulting in a large amount of irrelevant background information being mixed into the prototype representation. This leads to severe prototype bias when supporting samples are extremely limited. Furthermore, prototype generation methods based on simple averaging or summation lack robustness against noisy samples in the support set and inaccurately labeled regions. A few low-quality support samples can significantly reduce the representativeness of the category prototypes, leading to inaccurate similarity calculations between the query image and the prototype, and resulting in widespread false positives and false negatives in defect segmentation. The potential of dual-branch networks in multi-scale feature interaction and channel-level feature decoupling has not been fully explored, and an adaptive prototype calibration mechanism suitable for small sample scenarios needs to be proposed. Summary of the Invention

[0003] This paper presents a method and system for identifying surface defects of products based on image recognition in small samples. Under the condition of very few labeled samples, it alleviates the problems of prototype expression bias and insufficient utilization of scale information by multi-scale feature decoupling and prototype adaptive calibration, thereby improving the accuracy and robustness of product surface defect region segmentation.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a method for identifying surface defects of products using a small sample size based on image recognition. This method can be used to accurately locate and classify various defects on the product surface under small sample conditions. The method acquires a query image of the product surface to be identified and a support set containing a small number of labeled samples of known defect categories. As a preferred technical solution of the present invention, an industrial camera captures the original image of the product surface, which is then normalized in brightness and scaled to obtain the query image. The support set is formed by selecting a preset number of labeled samples from a historical defect database for each defect category. The number of samples under each category is the same and generally does not exceed ten. Each labeled sample contains an image of the product surface and a corresponding defect area mask label, thereby ensuring the stability of the identification with very few samples.

[0005] This invention extracts multi-scale feature maps from the query image and the support set using a dual-branch feature extraction network, and decouples these multi-scale feature maps at the channel level to obtain several feature subspaces. The first branch of the dual-branch feature extraction network receives the query image, and the second branch receives each labeled sample image from the support set. The two branches share network parameters, ensuring that the query and support set features are expressed in the same feature space. Each branch sequentially goes through multiple convolutional stages, with each stage outputting a feature map of one scale, thus obtaining a set of multi-scale feature maps for the query image and the support set. Preferably, the first and second branches each contain four convolutional stages, each consisting of a convolutional layer, a batch normalization layer, and an activation function layer connected sequentially, with the size of the output feature map halved in each stage. As a further improvement, after the first branch outputs the multi-scale feature map for each convolutional stage, it simultaneously passes it to the corresponding feature map in the second branch for element-wise addition, resulting in a fused multi-scale feature map for the support set, enhancing the adaptability of the support set features to the scale distribution of the query image. For the query image feature map and support set feature map at each scale, the feature map is uniformly divided into a preset number of feature subspaces according to the channel dimension. Each feature subspace contains feature responses from several consecutive channels. When performing channel segmentation, preferably, the number of channels in each feature subspace is obtained by dividing the total number of channels in the feature map by the preset number of segments. Starting from the first channel of the feature map, multiple consecutive channel groups are sequentially extracted with this number of channels as the step size. Each channel group is treated as an independent feature tensor, and the original spatial dimension remains unchanged during the segmentation process, thus decoupling feature responses with different semantic attributes. As a preferred implementation, each feature subspace corresponds to an independent learning scale factor, used to adaptively adjust the amplitude of the feature responses within that subspace, strengthening the feature subspace that plays a key role in defect detection and suppressing redundant responses unrelated to defects, thereby improving the discriminative power of feature representation.

[0006] This invention employs a learnable prototype calibration module to perform weighted aggregation of support set features in each feature subspace, obtaining a calibration prototype for each defect category. For the current feature subspace, feature vectors of all support samples belonging to the same defect category are extracted within that feature subspace. The similarity between each support sample feature vector and the feature vectors of other support samples in that category is calculated using the attention calculation unit in the prototype calibration module, yielding a weight coefficient for each support sample. Then, each support sample feature vector is multiplied by its corresponding weight coefficient, summed, and divided by the total number of support samples in that category to obtain the calibration prototype for that defect category in the current feature subspace. This process traverses all feature subspaces and all defect categories, generating a calibration prototype for each defect category in each feature subspace. As a preferred implementation, the attention calculation unit employs a multi-head attention mechanism, mapping each support sample feature vector to multiple attention heads, calculating similarity separately, and then concatenating the results to obtain the final weight coefficient. This captures the interrelationships between samples from different representation subspaces, making the weight estimation more robust. When calculating similarity, a target feature vector is selected from all supporting sample feature vectors of the current defect category. The dot product of this target feature vector and the feature vectors of each of the remaining supporting samples in the same category is calculated to obtain multiple raw similarity scores. All raw similarity scores are normalized so that the sum of the weight coefficients of all supporting samples in the same category is one. The normalized score is then used as the weight coefficient of the target feature vector. The prototype calibration module can also incorporate a learnable temperature parameter to scale the raw similarity scores when the attention calculation unit calculates similarity, controlling the concentration of weight distribution and helping to stabilize the prototype in cluttered backgrounds. As a preferred enhancement method, after obtaining the calibration prototype for each defect category in each feature subspace, the defect region mask of the support set labeled samples is used to spatially constrain the calibration prototype. This means that the response value of the corresponding background region in the calibration prototype is forced to be zero, retaining only the response value of the corresponding defect region. This avoids background information from contaminating the prototype representation and improves the accuracy of defect region localization.

[0007] This invention calculates the correlation response map between the feature subspace of a query image and the calibration prototype. For each feature subspace of the query image, a query feature tensor corresponding to that feature subspace is extracted. This query feature tensor has spatial height and spatial width dimensions. For the query feature vector at each spatial location, the cosine similarity between it and the calibration prototype for each defect category in the same feature subspace is calculated, obtaining the similarity value for that spatial location to each defect category. As an alternative, cosine similarity can be replaced with learnable Mahalanobis distance, where the covariance matrix in the Mahalanobis distance is estimated by all sample feature vectors in the support set, making the similarity measure better adaptable to the feature distribution. The similarity values ​​of all defect categories at the same spatial location are aggregated to form a multi-channel response vector for that spatial location. The multi-channel response vectors of all spatial locations constitute the correlation response map.

[0008] After obtaining the correlation response map, it is used to segment defect regions and obtain defect identification results. A maximum value operation is performed on the correlation response map along the defect category dimension to obtain the maximum similarity value and its corresponding defect category index at each spatial location. The maximum similarity value at each spatial location is compared with a preset confidence threshold. When the maximum similarity value is greater than or equal to the confidence threshold, the spatial location is marked as a defect region corresponding to the defect category index. To improve the stability of the segmentation results, connected component analysis is performed on all spatial locations marked as defect regions to remove isolated regions with an area smaller than a preset area threshold, thereby obtaining the final defect segmentation mask and the category label for each defect region, effectively filtering out false alarms and improving the completeness of defect regions.

[0009] Based on the same inventive concept, this invention also provides a small-sample product surface defect identification system based on image recognition, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the aforementioned small-sample product surface defect identification method based on image recognition. This system can automatically and accurately identify and segment various defects on the product surface with very few labeled samples, adapting to the practical application needs of products with varied surface textures, diverse defect morphologies, and scarce samples.

[0010] The technical effects and advantages provided by the present invention in the above technical solution are as follows: A dual-branch feature extraction network extracts multi-scale feature maps from the query image and support set, and performs channel-level decoupling on each scale feature map to obtain several feature subspaces. This splits the originally aggregated channel responses into multiple independent feature subspaces. Each feature subspace corresponds to a continuous set of channels, and different subspaces capture the response patterns of the product surface under specific semantic attributes, distributing information such as the texture, edges, and geometry of defects into different subspaces for differentiated expression. The multi-scale feature maps themselves contain hierarchical information from local details to global contours. Channel decoupling at each scale maintains independent response capabilities to defect detail attributes at different spatial resolutions. This ensures that small defects are not submerged by the background in shallow, high-resolution feature subspaces, while large defects retain complete semantic contours in deep, low-resolution feature subspaces. Each feature subspace corresponds to an independent learning scale factor, which adaptively adjusts the amplitude of the feature responses within that subspace. This avoids the problems of insufficient contribution from important subspaces and increased interference from redundant subspaces that occur when directly using equal amplitudes to calculate similarity between features in different subspaces. This multi-scale decoupling structure enables subsequent prototype computation and similarity matching to utilize richer and purer feature representations. Even with a very small number of support samples, it can still acquire discriminative features sufficient to distinguish defective regions from the background, thereby improving the precision of defect segmentation and adaptability to scale changes. A learnable prototype calibration module is used to perform weighted aggregation of support set features in each feature subspace. The similarity between feature vectors of support samples within the same category is evaluated through an attention computation unit, dynamically generating the weight coefficient of each support sample. For support samples whose features differ significantly from other samples within the category, their weights are automatically suppressed, reducing the contribution of low-quality or anomalous support samples in the calibration prototype. Support samples that are highly consistent with other samples are given higher weights, ensuring that the generated calibration prototype is closer to the center of the true feature distribution of the category. A multi-head attention mechanism is introduced into the attention computation, mapping support sample features to multiple attention heads, calculating similarity separately, and then concatenating the results to obtain the weight coefficients. This allows the weights to jointly consider the consistency between samples from multiple subspaces, avoiding the oversensitivity of a single similarity metric to feature perturbations. When computing the calibration prototype, spatial constraints are applied after weighted aggregation using existing defect region masks from the support set. This forces the response values ​​of corresponding background regions in the calibration prototype to zero, retaining only the responses of defect regions. This spatial constraint operation completely eliminates background information contamination of the prototype, allowing it to encode only the features of the defects themselves. When calculating the correlation response with query image features, similar background regions are not mistakenly activated as defects, and a more accurate correspondence can be established between foreground defect regions and the prototype. A learnable temperature parameter scales the original similarity score, adaptively adjusting the steepness of the similarity distribution during training, making the class distinction boundaries clearer in small sample sizes.Through the synergistic effect of the above weighted aggregation, spatial constraints and temperature regulation, even under the extreme condition of only a few supporting samples in each class, a stable and highly discriminative calibration prototype can still be obtained. The calculated correlation response map can accurately depict the location and shape of defects. Finally, the defect area in the segmentation mask is complete and has clear boundaries, and it shows good recognition effect for surface defects with extremely small size or varied shape. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0012] Figure 1 This is a flowchart of a small-sample product surface defect identification method based on image recognition. Figure 2 This is a schematic diagram of a few-sample defect detection process based on bi-branch feature extraction and multi-head attention prototype calibration; Figure 3 This is a flowchart of correlation response map calculation and defect segmentation based on the feature subspace calibration prototype; Figure 4 This is a schematic diagram of calibration prototype generation based on attention mechanism and spatial constraints; Figure 5 This is a flowchart of a small-sample product surface defect identification method based on image recognition. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] See Figure 1This invention provides a method for identifying surface defects in products using a small sample size based on image recognition. This method constructs a dual-branch feature extraction network to process the query image to be detected and the support set containing a small number of labeled samples separately. Channel-level decoupling of the feature maps is performed in a multi-scale feature space, forming multiple feature subspaces. Within each feature subspace, a learnable prototype calibration module is used to weighted aggregate the features of the support set, generating calibration prototypes for each defect category. Based on this, a correlation response map between the query image feature subspace and the calibration prototype is calculated, and pixel-level segmentation of the defect region is performed based on this correlation response map, thereby achieving accurate identification of product surface defects under small sample conditions.

[0015] Example 1: In specific implementation, please refer to Figure 2 The process involves acquiring a query image of the product surface to be identified and a support set containing a small number of labeled samples with known defect categories. The original image of the product surface is acquired using an industrial camera. Brightness normalization is performed on the original image, linearly mapping the image pixel values ​​to a standardized distribution with a mean of zero and a variance of one to suppress interference introduced by illumination variations. The brightness-normalized image is then scaled to adjust the image spatial resolution to a preset fixed size, resulting in the query image. This preset fixed size is determined based on the input requirements of the dual-branch feature extraction network. From a historical defect database, a preset number of labeled samples are selected for each defect category. Each labeled sample contains an image of the product surface and a corresponding defect area mask label. The defect area mask label is a binary image of the same size as the product surface image, with a defect area pixel value of 1 and a background area pixel value of 0. All selected labeled samples are combined into a support set, ensuring that the number of labeled samples for each defect category is the same and does not exceed ten.

[0016] The query image is input into the first branch of the dual-branch feature extraction network, and each labeled sample image from the support set is input into the second branch. The first and second branches share all network parameters, including the weight matrices and bias vectors of all convolutional layers. Each branch contains four convolutional stages, each consisting of a convolutional layer, a batch normalization layer, and an activation function layer connected sequentially. After the input image passes through the first convolutional stage, a first-scale feature map is output; after the first-scale feature map passes through the second convolutional stage, a second-scale feature map is output, with its height and width halved compared to the first-scale feature map; and so on, with the fourth convolutional stage outputting a fourth-scale feature map. Thus, the query image branch obtains a multi-scale feature map set composed of the first-scale query feature map, the second-scale query feature map, the third-scale query feature map, and the fourth-scale query feature map, while the support set branch obtains a multi-scale feature map set composed of the first-scale support feature map, the second-scale support feature map, the third-scale support feature map, and the fourth-scale support feature map.

[0017] For each scale of the query image feature map and support set feature map, the feature map is uniformly divided into a predetermined number of feature subspaces according to the channel dimension. Specifically, the total number of channels in the feature map is obtained, and the total number of channels is divided by the predetermined number of divisions; the quotient is the number of channels in each feature subspace. Starting from the first channel of the feature map, multiple consecutive channel groups are sequentially extracted with the number of channels in the feature subspace as the step size, and each channel group is considered a feature subspace. For all channels within each feature subspace, its spatial height and spatial width dimensions are kept unchanged, forming an independent feature tensor. In the predetermined number of feature subspaces, each feature subspace corresponds to an independent learning scale factor. The learning scale factor is a scalar parameter that is updated during training using the backpropagation algorithm to adaptively adjust the amplitude of all feature responses within its respective feature subspace.

[0018] For the current feature subspace, extract the feature vectors of all support samples belonging to the same defect category in the current feature subspace. This includes all samples belonging to the defect category. Support samples in feature subspace The eigenvectors under the following are denoted as the set. ,in, Indicates the category of defect The total number of supporting samples is determined. The feature vector set is then fed into the attention calculation unit in the prototype calibration module, which employs a multi-head attention mechanism. In this mechanism, each supporting sample feature vector is projected onto multiple attention heads via a linear mapping, and each attention head independently calculates its similarity. For a given attention head, the similarity is calculated based on the defect category. Select a target feature vector from the set of feature vectors Calculate the target feature vector Compared with the feature vectors of each other support sample in the same category The dot product is used to obtain the original similarity score. The prototype calibration module has a built-in learnable temperature parameter. The temperature parameter It is a positive real number, optimized along with the network parameters during training. The temperature parameter is used. The original similarity score is scaled to obtain the scaled similarity score. For the target feature vector The target feature vector is normalized by calculating the scaled similarity scores of all support sample feature vectors except itself. This normalization process uses the softmax function, ensuring that the sum of the weight coefficients of all support samples within the same category is one. The normalized score is then the target feature vector. Corresponding weight coefficients The calculation formula is expressed as:

[0019] in, Indicates the category of defect In the characteristic subspace Next The weight coefficients of each supporting sample, This represents a learnable temperature parameter. Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector Indicates the category of defect The total number of supporting samples This represents an exponential function with the natural constant as its base. The above calculations are performed independently for each attention head, resulting in weight coefficients for multiple attention heads. These weight coefficients are then concatenated to form the final weight coefficients under the multi-head attention mechanism.

[0020] Multiply each support sample feature vector by its corresponding weight coefficient to obtain a weighted feature vector. Summate all weighted feature vectors and then divide the sum by the defect category. Total number of supporting samples in the current feature subspace Obtain the defect category In the characteristic subspace The initial calibration prototype is obtained by traversing all feature subspaces and all defect categories to obtain the initial calibration prototype for each defect category in each feature subspace.

[0021] After obtaining the initial calibration prototype, spatial constraints are applied to the initial calibration prototype for each defect category in each feature subspace using the defect region mask provided by each labeled sample in the support set. The defect region mask is downsampled to the same resolution as the spatial size of the initial calibration prototype using bilinear interpolation. For each position of the initial calibration prototype in the spatial dimension, if the value of the downsampled defect region mask at that position is 0, then all channel response values ​​at the corresponding position of the initial calibration prototype are forced to zero; if the value of the downsampled defect region mask at that position is 1, then the response value at the corresponding position of the initial calibration prototype is left unchanged. After the spatial constraint processing, the final calibration prototype is obtained, which aggregates only the feature representations of defect regions in the support samples and suppresses the mixing of information from background regions.

[0022] Example 2: In specific implementation, please refer to Figure 3 After obtaining the calibration prototype for each defect category in each feature subspace, the correlation response map between the query image feature subspace and the calibration prototype is calculated.

[0023] For each feature subspace of the query image, the query feature tensor corresponding to that feature subspace is extracted from the multi-scale feature map set of the query image. The query feature tensor has a spatial height dimension. and spatial width dimension In every spatial location The above corresponds to one A dimensional query feature vector, where This indicates the number of channels contained in each feature subspace after the preset segmentation. The value range is 1 to , The value range is 1 to .

[0024] For each spatial location in the query feature tensor Query feature vectors The similarity between the query feature vector and the calibration prototype of each defect category in the same feature subspace is calculated. The similarity is cosine similarity, calculated as follows: [The query feature vector is then compared with the calibration prototype of each defect category in the same feature subspace.] With Defect Category In the characteristic subspace The calibration prototype Perform a dot product operation, and then divide the dot product result by the query feature vector. L2 norm and calibration prototype The product of the L2 norms is the quotient value, which is the query feature vector. For defect categories cosine similarity value The calculation formula is expressed as:

[0025] in, Indicates spatial location The query feature vector on the defect category The cosine similarity value, Indicates spatial location Query feature vectors on Indicates the category of defect In the characteristic subspace The calibration prototype below, This represents the vector dot product operation. Represents L2 norm operations. This indicates a scalar multiplication operation.

[0026] In other implementations, the cosine similarity is replaced by a learnable Mahalanobis distance. When using Mahalanobis distance as a metric, all support samples are extracted from the support set in the feature subspace. The feature vectors below form the support set feature matrix, and the covariance matrix is ​​estimated based on the support set feature matrix. Regarding spatial location Query feature vectors and defect categories calibration prototype Mahalanobis distance By calculating the query feature vector With calibration prototype The difference vector, and the difference vector with the covariance matrix. The inverse matrix is ​​obtained by performing a quadratic form operation. The Mahalanobis distance is then calculated. The negative value is then transformed into a similarity value through an exponential function mapping.

[0027] For each spatial location The similarity values ​​for all defect categories at the same location are aggregated along the defect category dimension to form a multi-channel response vector for that spatial location. The number of channels in the multi-channel response vector equals the total number of known defect categories. The spatial height dimension... and spatial width dimension The multi-channel response vectors of all spatial locations are combined according to their original spatial arrangement to form a correlation response map, wherein the tensor dimension of the correlation response map is [missing information]. ,in This indicates the total number of defect categories.

[0028] After obtaining the relevant response map, a maximum value operation is performed on the defect category dimension of the relevant response map. For each spatial location... Iterate through all defect category channels, select the channel with the highest similarity value, and record that highest similarity value. and the defect category index corresponding to this channel. The maximum similarity value at each spatial location. Compared with the preset confidence threshold Compare the results. (Preset confidence threshold) It is a pre-defined constant, ranging from 0 to 1, and its specific value is determined through segmentation accuracy evaluation on an offline validation set. If the maximum similarity value... Greater than or equal to the confidence threshold Then mark the spatial location as the defect category index. The corresponding defective area is assigned a semantic label, and the semantic label value is equal to the defect category index. If the maximum similarity value Less than the confidence threshold If the location is not specified, then the spatial location is marked as a background area and assigned a semantic label value of 0.

[0029] After labeling all spatial locations, connected component analysis is performed on all spatial locations of the labeled defective regions. Connected component analysis employs a two-pass scanning method or a region-growing-based connected component labeling algorithm to identify connected components formed by sets of defective pixels with the same semantic label and spatial adjacency. For each identified connected component, the total number of spatial locations contained within the connected component is counted as the connected component area. The area of ​​each connected component is then compared with a preset area threshold. Comparison, preset area threshold It is a pre-set constant, set based on the minimum acceptable defect size for product surface defect detection. It applies to areas smaller than a preset area threshold. Connected components are removed, and components with areas smaller than a preset area threshold are removed. The semantic labels of all spatial locations within the connected components are reset to 0, i.e., corrected to background regions. After connected component area filtering, the retained connected components constitute the final defect segmentation mask. Each retained connected component corresponds to a defect region, and the category label of the defect region is determined by the defect category index corresponding to the spatial location within the connected component. The defect segmentation mask and the category label of each defect region are output as the defect recognition result.

[0030] Example 3: In specific implementation, please refer to Figure 4 The prototype calibration module calculates the calibration prototype for each defect category in each feature subspace. The following is an explanation of the detailed data processing procedure.

[0031] The prototype calibration module includes an attention calculation unit. For a given feature subspace... and specified defect categories The attention calculation unit receives data that falls under the defect category. All supporting samples in the feature subspace Based on the feature vectors below, construct a feature vector set. Let the total number of supporting sample feature vectors in the feature vector set be denoted as . , That is, the category of defects The total number of supporting samples. Each supporting sample feature vector in the feature vector set is selected sequentially as the target feature vector.

[0032] For the currently selected target feature vector , where subscript Indicates the category of defect subscript Representing the characteristic subspace subscript The index of the target feature vector in the feature vector set is used to calculate the target feature vector. And each of the support sample feature vectors in the feature vector set except itself The dot product is used as the original similarity score. Learnable temperature parameters. It is a positive scalar, a temperature parameter. The initial value is set to 1.0, and the temperature parameter is used during training. The network is optimized and updated using the backpropagation algorithm along with other network parameters. Temperature parameters are utilized. Scaling is performed on each original similarity score, that is, scaling the original similarity score relative to the temperature parameter. Multiplying yields the scaled similarity score. This applies to the target feature vector. The corresponding scaled similarity scores are normalized using the softmax function, ensuring that the sum of the weight coefficients of all supporting sample feature vectors (excluding the target feature vector itself) in the feature vector set is one. After normalization, each supporting sample feature vector... The corresponding numerical value is the target feature vector. Support sample feature vectors from different perspectives The weighting coefficients. Preserving the target feature vector. Its own weight coefficient is the normalized target feature vector The weights are calculated from the values ​​themselves. These weight coefficients are denoted as... , indicating in the characteristic subspace Defect Category China and Israel When the nth support sample feature vector is used as the target feature vector, the nth support sample feature vector is used as the target feature vector. The weight coefficients of the feature vectors of each supporting sample. The calculation formula is expressed as:

[0033] in, Indicates the category of defect In the characteristic subspace The following are the first When the nth support sample feature vector is used as the target feature vector, the nth support sample feature vector is used as the target feature vector. The weight coefficients of the feature vectors of each supporting sample. This represents a learnable temperature parameter. Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector Indicates the category of defect In the characteristic subspace Next Each supporting sample feature vector This represents the dot product operation of vectors. This represents an exponential function with the natural constant as its base. Indicates the category of defect The total number of supporting samples.

[0034] For each supporting sample feature vector, the above operation is performed as a target feature vector, resulting in multiple sets of weight coefficients. The weight coefficients corresponding to all target feature vectors are then fused, with the fusion method being to combine the weights of the same first-order sample into the second-order sample. The average of the weight coefficients of each support sample feature vector under all different target feature vectors is calculated, and this average is used as the weight coefficient of the first support sample feature vector. The final weight coefficients of the feature vectors of each support sample Each support sample feature vector is compared with its corresponding final weight coefficient. Multiply the results to obtain a weighted feature vector. Summate all weighted feature vectors element-wise, and then divide the sum by the total number of support samples. The result obtained is the defect category. In the characteristic subspace The initial calibration prototype.

[0035] After obtaining the initial calibration prototype, read the supported concentrated defect categories. The defect region mask is provided by the corresponding labeled sample. The defect region mask is a binary image with the same spatial dimensions as the labeled sample image. In the defect region mask, a pixel value of 1 indicates a defect region, and a pixel value of 0 indicates a background region. The defect region mask is downsampled using bilinear interpolation to ensure that the spatial height and width dimensions of the downsampled defect region mask are consistent with those of the initial calibration prototype. For each position in the spatial dimension of the initial calibration prototype, if the pixel value of the downsampled defect region mask at that position is 0, then all channel response values ​​at the corresponding spatial position in the initial calibration prototype are forcibly set to zero; if the pixel value of the downsampled defect region mask at that position is 1, then the response value at the corresponding spatial position in the initial calibration prototype remains unchanged. After the above spatial constraint processing, the defect category is obtained. In the characteristic subspace The calibration prototype below. This calibration prototype only aggregates the feature responses of the real defect areas in the supporting samples, eliminating the interference of background area feature responses.

[0036] Example 4: In specific implementation, please refer to Figure 5 A small-sample product surface defect identification system based on image recognition includes a memory and a processor. The memory stores a computer program containing multiple program modules, each responsible for executing a step of the image recognition-based small-sample product surface defect identification method. The processor calls and executes the computer program in the memory to implement the complete defect identification process.

[0037] When executing the computer program, the processor performs the data acquisition step. The processor sends an acquisition command to the industrial camera and receives the original image of the product surface to be identified returned by the camera. The processor performs brightness normalization processing on the original image, subtracting the mean of all pixel grayscale values ​​in the original image from the grayscale value of each pixel, and then dividing the result by the standard deviation of all pixel grayscale values ​​in the original image, so that the pixel value distribution of the brightness-normalized image has zero mean and unit variance. The processor performs size scaling processing on the brightness-normalized image, adjusting the image spatial resolution to a preset fixed size using a bilinear interpolation algorithm to obtain the query image. Simultaneously, the processor accesses the historical defect database, reading labeled samples one by one according to defect category. The number of labeled samples read for each defect category is a preset number, not exceeding ten. Each labeled sample contains an image of the product surface and a corresponding defect area mask label. In the defect area mask label, a pixel value of 1 indicates a defect area, and a pixel value of 0 indicates a background area. The processor combines all the labeled samples it reads into a support set and ensures that the number of labeled samples is the same for each defect category.

[0038] After the data acquisition step is completed, the processor executes the two-branch feature extraction step. The processor inputs the query image into the first branch of the two-branch feature extraction network and inputs each labeled sample image from the support set into the second branch of the two-branch feature extraction network. The first and second branches of the two-branch feature extraction network each contain four convolutional stages, each consisting of a convolutional layer, a batch normalization layer, and an activation function layer connected sequentially. Convolutional stages with the same position number in the first and second branches share all network parameters, including the weight matrix and bias vector of the convolutional layer, and the scaling and offset parameters of the batch normalization layer. The spatial height and spatial width of the feature map output by each convolutional stage are halved compared to the input feature map of that convolutional stage. After the query image is processed sequentially through the four convolutional stages of the first branch, it outputs a first-scale query feature map, a second-scale query feature map, a third-scale query feature map, and a fourth-scale query feature map. After each labeled sample image from the support set is processed sequentially through the four convolutional stages of the second branch, it outputs a first-scale support feature map, a second-scale support feature map, a third-scale support feature map, and a fourth-scale support feature map. After the first branch executes the multi-scale feature map output of each convolution stage, the processor passes the query image multi-scale feature map output by the first branch to the corresponding convolution stage in the second branch, and performs an element-wise addition operation with the support set multi-scale feature map output by the corresponding convolution stage of the second branch to obtain the fused support set multi-scale feature map.

[0039] After the dual-branch feature extraction step is completed, the processor performs a channel-level decoupling step. For each scale feature map of the query image and each scale feature map of the fused support set, the processor obtains the total number of channels in the feature map, divides the total number of channels by a preset number of segments, and uses the quotient as the number of channels in each feature subspace. Starting from the first channel of the feature map, the processor sequentially extracts multiple consecutive channel groups with the number of channels in the feature subspace as the step size, treating each channel group as a feature subspace. For all channels within each feature subspace, the processor maintains the spatial height and spatial width dimensions unchanged, forming an independent feature tensor. The processor assigns an independent learning scale factor to each feature subspace. The learning scale factor is a trainable scalar parameter, updated during the training phase using the backpropagation algorithm, and used as a fixed value after training convergence during the inference phase to scale and adjust the amplitude of all feature responses within that feature subspace.

[0040] After the channel-level decoupling step is completed, the processor performs prototype calibration and spatial constraint steps. For each feature subspace and each defect category, the processor extracts the feature vectors of all supporting samples belonging to the same defect category within that feature subspace, forming a feature vector set. The processor then feeds this feature vector set into the attention calculation unit within the prototype calibration module. The attention calculation unit employs a multi-head attention mechanism, projecting each supporting sample feature vector onto multiple attention heads via a linear mapping. Each attention head performs its computation independently. Within each attention head, the processor selects a target feature vector from the feature vector set and calculates the dot product between the target feature vector and the feature vectors of each of the remaining supporting samples in the same category, obtaining the original similarity score. The prototype calibration module incorporates a learnable temperature parameter. Temperature parameters The initial value is set to 1.0, and the temperature parameter is used during training. Optimization and adjustment are achieved through backpropagation algorithm. The processor utilizes temperature parameters. The original similarity score is scaled, and then softmax normalization is performed on the scaled similarity score to obtain the weight coefficients corresponding to the target feature vectors. The processor fuses the weight coefficients obtained from different target feature vector perspectives within the same attention head to obtain the final weight coefficients for each support sample feature vector. The processor multiplies each support sample feature vector by its corresponding final weight coefficient to obtain a weighted feature vector. The sum of all weighted feature vectors is then divided by the total number of support samples in the defect category to obtain the initial calibration prototype for the defect category in the current feature subspace. After traversing all feature subspaces and all defect categories, the processor reads the defect region mask of the labeled samples in the support set, downsamples the defect region mask to the same spatial size as the initial calibration prototype, and forces all channel response values ​​in the initial calibration prototype corresponding to the spatial position of the defect region mask with a pixel value of 0 after downsampling to zero, while the response values ​​of the spatial position with a pixel value of 1 remain unchanged, thus obtaining the calibration prototype for each defect category in each feature subspace.

[0041] After the prototype calibration and spatial constraint steps are completed, the processor executes the relevant response map calculation step. For each feature subspace of the query image, the processor extracts the corresponding query feature tensor, which has spatial height and spatial width dimensions. For each query feature vector at a spatial location, the processor calculates the cosine similarity between the query feature vector and the calibration prototype for each defect category in the same feature subspace, obtaining the similarity value for that spatial location to each defect category. The cosine similarity is calculated by performing a dot product operation between the query feature vector and the calibration prototype, and then dividing by the product of the L2 norm of the query feature vector and the L2 norm of the calibration prototype. Optionally, cosine similarity is replaced by Mahalanobis distance. The covariance matrix in the Mahalanobis distance is estimated by all sample feature vectors in the support set. The Mahalanobis distance is negative and transformed into a similarity value through an exponential function mapping. The processor aggregates the similarity values ​​of all defect categories at the same spatial location along the defect category dimension to form a multi-channel response vector, and combines the multi-channel response vectors of all spatial locations into a relevant response map.

[0042] After the relevant response map calculation step is completed, the processor performs a defect region segmentation step. The processor performs a maximum value operation on the defect category dimension of the relevant response map, extracting the maximum similarity value and the corresponding defect category index for each spatial location. The processor compares the maximum similarity value of each spatial location with a preset confidence threshold. The confidence threshold is a pre-defined constant ranging from 0 to 1, determined through segmentation accuracy evaluation on an offline validation set. If the maximum similarity value is greater than or equal to the confidence threshold, the processor marks the spatial location as the defect region indicated by the corresponding defect category index; if the maximum similarity value is less than the confidence threshold, the processor marks the spatial location as a background region. The processor performs connected component analysis on all spatial locations marked as defect regions. The connected component analysis uses a two-pass scanning method to identify connected components composed of spatially adjacent pixels with the same defect category index and calculates the area of ​​each connected component. The processor compares the area of ​​each connected component with a preset area threshold, which is set based on the minimum acceptable defect size for product surface defect detection. Connected components with areas smaller than the preset area threshold are removed, and their spatial locations are corrected to background areas. The processor finally outputs a defect segmentation mask and a category label for each defect region, completing the identification of product surface defects under small sample conditions.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A small sample product surface defect identification method based on image recognition, characterized in that, Includes the following steps: Obtain a query image of the surface of the product to be identified and a support set of a small number of labeled samples containing known defect categories; A dual-branch feature extraction network is used to extract multi-scale feature maps from the query image and the support set respectively, and channel-level decoupling is performed on the multi-scale feature maps to obtain several feature subspaces; A learnable prototype calibration module is used to perform weighted aggregation of support set features in each feature subspace to obtain a calibration prototype for each defect category. The correlation response map between the feature subspace of the query image and the calibration prototype is calculated, and the defect region is segmented using the correlation response map to obtain the defect identification result. 2.The image recognition-based small sample product surface flaw identification method according to claim 1, wherein, The process involves extracting multi-scale feature maps from the query image and support set using a dual-branch feature extraction network, and then performing channel-level decoupling on the multi-scale feature maps to obtain several feature subspaces. Specifically, this includes: The query image is input into the first branch of the dual-branch feature extraction network, and each labeled sample image in the support set is input into the second branch of the dual-branch feature extraction network, wherein the first branch and the second branch share network parameters; In each branch, multiple convolutional stages are sequentially passed, and each convolutional stage outputs a feature map of one scale, thereby obtaining a multi-scale feature map set of the query image and a multi-scale feature map set of the support set. For each scale, the feature map of the query image and the feature map of the support set are evenly divided into a preset number of feature subspaces according to the channel dimension. Each feature subspace contains feature responses of several consecutive channels. 3.The image recognition-based small sample product surface flaw identification method according to claim 2, characterized in that, Each feature subspace in the preset number of feature subspaces corresponds to an independent learning scale factor, which is used to adaptively adjust the amplitude of the feature response within that subspace. 4.The image recognition-based small sample product surface flaw identification method according to claim 2, characterized in that, The step of uniformly dividing the feature map into a preset number of feature subspaces according to the channel dimension specifically includes: Obtain the total number of channels in the feature map, and divide the total number of channels by a preset number of segments to obtain the number of channels in each feature subspace; Starting from the first channel of the feature map, multiple consecutive channel groups are extracted sequentially with the number of channels in each feature subspace as the step size, and each channel group is a feature subspace. For all channels within each feature subspace, keep their spatial dimensions unchanged to form an independent feature tensor. 5.The image recognition-based small sample product surface flaw identification method according to claim 1, wherein, The learnable prototype calibration module performs weighted aggregation of support set features in each feature subspace to obtain a calibration prototype for each defect category, specifically including: For the current feature subspace, extract the feature vectors of all support samples belonging to the same defect category in that feature subspace; The similarity between the feature vector of each support sample and the feature vector of other support samples in the same category is calculated by the attention calculation unit in the prototype calibration module, and the weight coefficient of each support sample is obtained. Multiply each support sample feature vector by its corresponding weight coefficient, sum the results, and then divide by the total number of support samples in that category to obtain the calibration prototype of the defect category in the current feature subspace. Traverse all feature subspaces and all defect categories to obtain the calibration prototype for each defect category in each feature subspace.

6. The image recognition-based small sample product surface flaw identification method according to claim 5, characterized in that, The attention calculation unit adopts a multi-head attention mechanism, which maps each support sample feature vector to multiple attention heads, calculates the similarity of each head, and then concatenates them to obtain the final weight coefficient.

7. The image recognition-based small sample product surface flaw identification method according to claim 5, characterized in that, The calculation of the similarity between the feature vector of each support sample and the feature vectors of other support samples in the same category, to obtain the weight coefficient of each support sample, specifically includes: Select a target feature vector from all supporting sample feature vectors of the current defect category; Calculate the dot product between the target feature vector and the feature vector of each of the remaining supporting samples in that category to obtain multiple raw similarity scores; All original similarity scores are normalized so that the sum of the weight coefficients of all supporting samples in the same category is one. The normalized scores are then used as the weight coefficients of the target feature vector.

8. The image recognition-based small sample product surface flaw identification method according to claim 1, characterized in that, The calculation of the correlation response map between the feature subspace of the query image and the calibration prototype specifically includes: For each feature subspace of the query image, extract the query feature tensor corresponding to that feature subspace, wherein the query feature tensor has a spatial height dimension and a spatial width dimension; For each query feature vector at a spatial location, calculate the cosine similarity between the query feature vector and the calibration prototype of each defect category in the same feature subspace to obtain the similarity value of that spatial location to each defect category. The similarity values ​​of all defect categories at the same spatial location are aggregated to form a multi-channel response vector for that spatial location. The multi-channel response vectors of all spatial locations constitute the correlation response map.

9. The image recognition-based small sample product surface flaw identification method according to claim 8, characterized in that, The cosine similarity is replaced by a learnable Mahalanobis distance, where the covariance matrix in the Mahalanobis distance is estimated by using the feature vectors of all samples in the support set.

10. A small sample product surface defect identification system based on image recognition, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image recognition-based small sample product surface defect identification method according to any one of claims 1 to 9.