Fastener defect automatic identification method based on deep learning

Abstract

The invention discloses a fastener defect automatic identification method based on deep learning, and the method comprises the following steps: S1, collecting fastener image data, and generating a standardized image sample; s2, executing edge feature enhancement and multi-scale filtering operation to generate an image feature map; s3, inputting into an improved TransNeXt model, and outputting a defect stage label, an identification confidence coefficient and a feature coding vector; s4, screening the position of the suspicious region, executing a local region decoding operation, and generating a defect response thermodynamic diagram; s5, extracting a bounding box, morphological characteristics and response intensity distribution of the maximum response region, and constructing a defect candidate region set; s6, non-defect areas are screened out, and a final defect recognition result is obtained; and S7, constructing a defect distribution diagram and a statistical information table, and outputting a defect type, a position coordinate, an area estimation value and an identification confidence level. According to the invention, high-precision automatic identification and structured information extraction of fastener defects are realized.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and computer vision technology, and in particular to an automatic fastener defect identification method based on deep learning. Background Technology

[0002] With the continuous upgrading of industrial manufacturing and precision testing technologies, the demand for intelligent identification and classification of minute surface defects in metal structural components is increasing, especially in fields such as aerospace, automotive manufacturing, and high-end equipment processing, where higher requirements are placed on the accuracy of surface defect identification, location accuracy, and type discrimination capabilities. Currently, commonly used image recognition methods are mostly based on two-dimensional visual information, employing standard convolutional neural networks or simple feature focusing strategies to identify and label defects. However, in actual production scenarios, these methods generally suffer from the following problems:

[0003] First, due to the complex surface texture of metallic materials, variable lighting conditions, and uneven distribution of defect morphology, traditional convolutional structures often struggle to cover defect regions at different scales, resulting in unsatisfactory recognition of small, blurry, or poorly defined defects, leading to missed detections and false detections. Second, existing models mostly employ fixed structures when performing multi-stage feature extraction, lacking modeling of dependencies between semantic levels. This easily leads to redundant feature information or missing key semantics, affecting the model's generalization ability and fine-grained classification performance. Furthermore, their stable tracking ability of defect regions in sequential images is limited, easily affected by image occlusion, slight defect deformation, or changes in shooting angle, causing unstable region recognition and limiting the reliability of defect statistical analysis and structured output.

[0004] Therefore, how to provide an automatic fastener defect identification method based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an automatic fastener defect identification method based on deep learning. This invention fully utilizes image feature map enhancement, an improved TransNeXt model, and defect response heatmap construction techniques. It details how, based on standardized image samples, geometric edge structure, texture details, and grayscale distribution features are extracted, combined with attention-guided coding, gated residual fusion, semantic consistency mapping, and high-response region screening, to achieve defect stage label determination, defect candidate region extraction, and final defect identification result output. It has the advantages of high recognition accuracy, accurate positioning, and adaptability to a wide range of scenarios.

[0006] An automatic fastener defect identification method based on deep learning according to an embodiment of the present invention includes the following steps:

[0007] S1. Acquire fastener image data and perform size normalization, illumination equalization and noise suppression processing on the image data to generate standardized image samples;

[0008] S2. Perform edge feature enhancement and multi-scale filtering operations on the standardized image samples to extract geometric edge structure, texture details and gray-level distribution features, and generate image feature maps.

[0009] S3. Input the image feature map into the improved TransNeXt model and output the defect stage label, recognition confidence and feature encoding vector; the improved TransNeXt model includes a hierarchical encoding structure, a dynamic mapping structure, a temporal feedback structure and residual correction rules;

[0010] S4. Based on the defect stage label and identification confidence level, filter the location of suspicious areas, and combine the feature encoding vector to perform local area decoding operation to generate a defect response heatmap;

[0011] S5. Extract the bounding box, morphological features and response intensity distribution of the maximum response region based on the defect response heatmap, and construct a set of defect candidate regions;

[0012] S6. Perform geometric morphology filtering, region connectivity judgment and response anomaly evaluation on the set of defect candidate regions to screen out non-defect regions and obtain the final defect identification result.

[0013] S7. Based on the final defect identification results, construct a defect distribution map and statistical information table, and output the defect type, location coordinates, area estimate and identification confidence level.

[0014] Preferably, S1 specifically comprises:

[0015] Acquire fastener surface image data, including RGB images taken from different angles, distances, and lighting conditions;

[0016] The image data is subjected to size normalization processing to unify the image size to a preset resolution range and maintain a consistent aspect ratio; the size-normalized image is subjected to illumination equalization processing, and the brightness distribution is corrected using the histogram equalization method.

[0017] Noise suppression processing is performed on the image after illumination equalization. Median filtering and bilateral filtering algorithms are used in combination to optimize high-frequency noise and edge detail preservation, respectively, and to remove background interference and imaging errors in image acquisition. The processed image is saved as a standard format image file to construct standardized image samples.

[0018] Preferably, S2 specifically includes:

[0019] Edge feature enhancement processing is performed on standardized image samples. Gradient operators are used to extract high-frequency edge information in the image, and an adaptive threshold segmentation method is combined to highlight the fastener outline and thread structure line.

[0020] Multi-scale filtering is performed on the edge-enhanced image, and Gaussian and Laplacian filter layers with different kernel sizes are constructed to extract features from the detail layer and the structural layer in a hierarchical manner.

[0021] In the multi-scale feature extraction process, the difference mapping between feature layers at each scale is calculated, and the local texture change region and the gray-scale distribution abnormal region are weighted and superimposed to generate a fused feature map; normalization and channel concatenation operations are performed on the fused feature map to form an image feature map.

[0022] Preferably, the improved TransNeXt model is as follows:

[0023] The improved TransNeXt model includes a hierarchical coding structure, a dynamic mapping structure, a temporal feedback structure, and a residual correction rule;

[0024] The image feature map is divided into channels using a hierarchical coding structure, and multiple scale branch paths are set. Each scale branch path uses asymmetric convolution kernels and dilated convolution structures to extract geometric edge features and local texture features. Skip connection paths are constructed to fuse the shallow and deep features of each scale branch to generate hierarchical coding results.

[0025] The dynamic mapping structure constructs an attention distribution map based on the hierarchical encoding results, introduces semantic relation vectors, performs weight guidance operations on the attention matrix, and generates a dependency mapping feature map that integrates semantic consistency and spatial relevance.

[0026] The temporal feedback structure establishes a cross-layer residual propagation path between each coding layer, calculates the difference between the initial predicted feature and the output feature of the previous layer, and generates a residual signal; the output feature of the previous layer and the residual signal are concatenated and input to the next layer, and the feature state is updated through a gated loop unit to form a feature expression sequence; the feature expression sequence has a time dependency.

[0027] The residual correction rule constructs a semantic consistency metric based on the residual signals of the preceding and following coding layers to determine the degree of deviation of the residual signal in semantic expression; if it is less than the preset similarity threshold, gradient adjustment and weight update operations are performed on the residual signal to generate a residual signal feature map.

[0028] Preferably, S3 specifically includes:

[0029] The image feature map is input into the hierarchical coding structure, and initial edge features and texture features are extracted on each scale branch path respectively; cross-layer feature fusion is performed through skip connection paths to generate hierarchical coding results;

[0030] The hierarchical encoding result is input into the dynamic mapping structure, and the dependency mapping feature map is calculated by combining the semantic relation vector. Then, the semantic consistency enhancement operation is performed based on the attention distribution map.

[0031] The dependency mapping feature map is input into a temporal feedback structure to construct a feature representation sequence; the feature representation sequence establishes cross-layer feature dependencies through residual signal splicing and gated update operations.

[0032] The residual correction rule is applied to the feature expression sequence to generate a residual signal feature map, the degree of semantic expression shift is determined, and gradient adjustment and weight update are performed.

[0033] The corrected feature representation sequence is input into the fully connected layer and the classification output layer to generate defect stage labels, identification confidence and feature encoding vectors; the feature encoding vectors are the semantic encoding results in the residual signal feature map.

[0034] Preferably, S4 specifically includes:

[0035] Based on the defect stage label, determine the location range of the fastener in the abnormal stage, and construct a confidence screening mask on the image feature map in combination with the recognition confidence. Extract the image area with the recognition confidence greater than the preset suspicious threshold as the suspicious area.

[0036] For the suspicious region, feature extraction is performed in the feature encoding vector, and a multi-channel convolutional structure is called to perform local semantic decoding to generate a region-level feature map; the region-level feature map is upsampled and the boundary is corrected to restore the feature map to the image scale space;

[0037] A confidence weight map is constructed by fusing the region decoding results in the image scale space with the original image structure-guided edge map.

[0038] The confidence weight maps are superimposed by channels to generate a defect response heatmap; the defect response heatmap includes regional response intensity distribution and location activation information.

[0039] Preferably, S5 specifically includes:

[0040] Perform channel fusion on the defect response heatmap to generate a single-channel response intensity map;

[0041] Local peak points of continuous regions are extracted from the single-channel response intensity map, and a response intensity threshold is set to filter high response regions; the high response region is a region with a pixel value greater than the response intensity threshold and forming a connected structure.

[0042] Calculate the bounding box of the circumscribed rectangle based on the high response region, record the center coordinates, width and height dimensions and rotation angle of the bounding box, and construct a candidate boundary set;

[0043] Extract the response intensity distribution of the region corresponding to each bounding box in the candidate boundary set, calculate the average response value, maximum response value and response variance, and obtain the response distribution characteristics;

[0044] By combining response distribution characteristics and bounding box geometry, a set of candidate defect regions is constructed; the set of candidate defect regions includes region boundary information, morphological description information and response statistical characteristics.

[0045] Preferably, the step of performing channel fusion on the defect response heatmap to generate a single-channel response intensity map specifically involves:

[0046] Normalization is performed on each channel of the defect response heatmap to standardize the pixel values ​​within the channel to a set range;

[0047] A pixel-wise weighted summation operation is performed on the normalized multi-channel heatmap, wherein the weighting coefficients are set according to the channel importance of the decoding convolution kernel corresponding to each channel;

[0048] The weighted summation fused image is used as a single-channel response intensity image, and the weighted response value of each pixel in all channel responses is retained;

[0049] Edge smoothing is performed on the single-channel response intensity map, and Gaussian filtering is used to suppress local noise fluctuations.

[0050] Output a single-channel response intensity map, which includes the fused response value of each image region in the multi-channel response.

[0051] Preferably, S6 specifically includes:

[0052] Perform geometric morphology analysis on the bounding box of each candidate region in the defect candidate region set, and calculate the aspect ratio, contour complexity and boundary smoothness;

[0053] Set geometric shape filtering rules to eliminate candidate regions that do not meet the set shape parameter range;

[0054] Perform connectivity judgment on the retained candidate regions, construct a connected graph structure, and detect whether there are isolated pixel blocks or broken connection regions in the region;

[0055] Candidate regions with connectivity less than a preset connectivity threshold are removed, and a set of connected regions is constructed.

[0056] For each region in the set of connected regions, a response anomaly index is calculated, which includes the region's average response deviation, response gradient distribution, and response extreme value fluctuation; candidate regions with a response anomaly index less than a preset response anomaly threshold are removed.

[0057] Candidate regions that simultaneously meet the geometric shape filtering rules, connectivity judgment criteria, and response anomaly evaluation criteria are taken as the final defect identification results.

[0058] Preferably, S7 specifically includes:

[0059] Extract the location coordinate information of each defect region in the final defect identification result, including the coordinates of the region center point and the coordinates of the bounding box vertices;

[0060] Based on the boundary range of the defect area and the pixel size ratio, the area estimate is calculated and expressed in a uniform unit.

[0061] Based on the label results of the corresponding defect areas in the classification output layer, label the defect type names and construct a defect type set;

[0062] The identification confidence level of the defective region in the feature expression sequence is statistically analyzed, and the identification confidence level value is recorded to form an identification confidence level set.

[0063] By integrating defect type, location coordinates, area estimate, and identification confidence level, a structured statistical information table is constructed;

[0064] Based on the defect location coordinates recorded in the statistical information table, draw defect label boundaries in the original image, overlay defect type and identification confidence level information, and construct a defect distribution map.

[0065] The beneficial effects of this invention are:

[0066] This invention addresses the issues of uneven illumination, noise interference, and loss of texture details in fastener surface images during acquisition by deeply fusing image feature enhancement with an improved TransNeXt model. It employs image standardization processing with size normalization, illumination equalization, and noise suppression, combined with edge feature enhancement and multi-scale filtering operations to extract geometric edge structure, texture details, and grayscale distribution features, generating high-quality image feature maps. In the feature encoding stage, a hierarchical encoding structure, dynamic mapping structure, temporal feedback structure, and residual correction rules are introduced. Multi-scale feature fusion and semantic consistency enhancement are achieved through semantic relation vectors and attention distribution maps, outputting defect stage labels and recognition confidence scores. The system incorporates feature encoding vectors; in the defect region analysis stage, it combines identification confidence to screen suspicious region locations, performs local region decoding and confidence weight fusion, and generates a defect response heatmap; in the candidate region extraction and screening stage, it employs continuous region local peak detection, geometric morphology filtering, region connectivity judgment and response anomaly assessment to eliminate non-defect regions and output the final defect identification result; in the result output stage, it constructs a defect distribution map and statistical information table, generates defect type, location coordinates, area estimate and identification confidence level, and achieves high-precision detection and structured output of surface defects of multiple types of fasteners, with the beneficial effects of high identification accuracy, strong anti-interference ability and adaptability to different complex working conditions. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a flowchart of a fastener defect automatic identification method based on deep learning proposed in this invention;

[0069] Figure 2 This is a schematic diagram of an automatic fastener defect identification method based on deep learning proposed in this invention. Detailed Implementation

[0070] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0071] refer to Figure 1-2 A method for automatic identification of fastener defects based on deep learning includes the following steps:

[0072] S1. Acquire fastener image data and perform size normalization, illumination equalization and noise suppression processing on the image data to generate standardized image samples;

[0073] S2. Perform edge feature enhancement and multi-scale filtering operations on the standardized image samples to extract geometric edge structure, texture details and gray-level distribution features, and generate image feature maps;

[0074] S3. Input the image feature map into the improved TransNeXt model and output the defect stage label, recognition confidence and feature encoding vector;

[0075] S4. Based on the defect stage label and identification confidence level, filter the location of suspicious areas, and combine the feature encoding vector to perform local area decoding operation to generate a defect response heatmap;

[0076] S5. Extract the bounding box, morphological features and response intensity distribution of the maximum response region based on the defect response heatmap, and construct a set of defect candidate regions;

[0077] S6. Perform geometric morphology filtering, region connectivity judgment and response anomaly evaluation on the set of defect candidate regions to screen out non-defect regions and obtain the final defect identification result.

[0078] S7. Based on the final defect identification results, construct a defect distribution map and statistical information table, and output the defect type, location coordinates, area estimate and identification confidence level.

[0079] This implementation standardizes and improves the quality of fastener image data by performing size normalization, illumination equalization, and noise suppression, effectively reducing interference caused by differences in acquisition conditions. Furthermore, it enhances the identifiability of defect areas in the image by extracting geometric edge structures, texture details, and grayscale distribution features through edge feature enhancement and multi-scale filtering. The image feature map is then input into an improved TransNeXt model, which, based on its hierarchical coding structure and semantic residual signal correction rules, outputs defect stage labels, recognition confidence scores, and feature encoding vectors, achieving deep semantic recognition of multiple types of defects. Further, based on the defect stage labels and recognition confidence scores, it filters... The system identifies suspicious areas and performs local region decoding operations using feature encoding vectors to generate a defect response heatmap, improving the accuracy of candidate region extraction. By extracting bounding boxes, morphological features, and response intensity distributions from the maximum response areas in the defect response heatmap, a set of defect candidate regions is constructed, enhancing the ability to capture responses to local details. Simultaneously, geometric morphology filtering, region connectivity judgment, and response anomaly assessment are performed to eliminate non-defective regions, ensuring the accuracy and stability of the final defect identification results. Finally, a defect distribution map and statistical information table are constructed, outputting defect type, location coordinates, area estimates, and identification confidence levels, improving the structured nature and decision support value of the defect results output.

[0080] In this embodiment, S1 specifically refers to:

[0081] Acquire fastener surface image data, including RGB images taken from different angles, distances, and lighting conditions;

[0082] The image data is subjected to size normalization processing to unify the image size to a preset resolution range and maintain a consistent aspect ratio; the size-normalized image is subjected to illumination equalization processing, and the brightness distribution is corrected using the histogram equalization method.

[0083] Noise suppression processing is performed on the image after illumination equalization. Median filtering and bilateral filtering algorithms are used in combination to optimize high-frequency noise and edge detail preservation, respectively, and to remove background interference and imaging errors in image acquisition. The processed image is saved as a standard format image file to construct standardized image samples.

[0084] In this embodiment, S2 specifically refers to:

[0085] Edge feature enhancement processing is performed on standardized image samples. Gradient operators, including the Sobel and Scharr operators, are used to extract high-frequency edge information from the image. The edge response map of the fastener image is obtained by calculating the gradient intensity in the horizontal and vertical directions. An adaptive threshold segmentation method is used to divide the edge response map into high-response and low-response regions. The adaptive threshold is dynamically set based on the mean and standard deviation of the pixel gradient distribution in the edge response map, obtained through fitting analysis of the pixel gradient change amplitude in different regions. High-confidence edge structures are extracted, and the fastener outline and thread structure lines are highlighted. Edge enhancement operations improve the contrast and clarity of structural information in the image.

[0086] Multi-scale filtering operations were performed on the edge-enhanced image, constructing Gaussian and Laplacian filter layers with kernel sizes of 3×3, 5×5, and 7×7, respectively. Gaussian filtering was used to extract smoothing features from the detail layer, while Laplacian filtering was used to extract sharpening features from the structure layer. The Gaussian filtering results at different scales were represented as smoothing layer groups, and the Laplacian filtering results at different scales were represented as structure layer groups. Response intensity normalization was performed on the smoothing and structure layer groups to obtain multi-scale texture maps and multi-scale structure maps. Through hierarchical analysis between different scales, the integrity of detail and structural features was improved.

[0087] In the multi-scale feature extraction process, the difference mapping between feature layers at each scale is calculated. The difference mapping is generated by a response difference fitting function at the same image coordinate position. The difference fitting function is constructed by the average of the squared differences of pixel values ​​between the multi-scale filtering results. The local texture change response map is obtained by fitting. The local texture change response map and the response map of the gray-level distribution abnormal area are weighted and superimposed. The weighting coefficient is dynamically generated according to the texture complexity and edge density index. The fused feature map is obtained by weighted superposition. The fused feature map has detailed distribution information, structural enhancement information and gray-level change sensitivity information, which improves the saliency expression ability of the defect area.

[0088] Normalization and channel concatenation operations are performed on the fused feature map. Normalization uses linear normalization to map pixel values ​​to the [0,1] interval. Channel concatenation superimposes response maps of different scales in dimensional order to form a unified image feature map. The image feature map serves as the input data for the subsequent defect recognition model and has the structural characteristics of multi-scale, multi-channel, and enhanced feature integration.

[0089] In this embodiment, the improved TransNeXt model is specifically as follows:

[0090] In the hierarchical coding structure, channel partitioning is achieved through channel slicing with a fixed stride, dividing the image feature map into multiple sub-channel sets proportionally. Each scale branch path constructs asymmetric convolutional kernel combinations such as 3×1, 1×3, 5×1, and 1×5 to extract orientation-sensitive edge features. The dilated convolutional structure uses different dilation rates to cover different receptive fields, enhancing responsiveness to texture-changing regions. Skip connection paths introduce inter-layer contextual information by performing feature concatenation and normalization operations between shallow and deep feature maps, improving coding stability and multi-scale fusion capabilities. The fused hierarchical coding results enhance the structural perception of local defects such as thread breakage and edge gaps.

[0091] In the dynamic mapping structure, the semantic relation vector is calculated based on the feature similarity between regions in the image. The similarity is calculated by fitting the cosine similarity formula, and the angle between each pixel and its neighboring features is measured by vectors. The weight guidance operation in the attention matrix realizes feature redistribution under the action of the semantic relation vector, focusing attention on regions with potential defect features. The dependency mapping feature map is obtained by performing a weighted matrix multiplication between the attention matrix and the hierarchical encoding result, which preserves the continuity of image structure in the spatial dimension and enhances defect sensitivity in the semantic dimension.

[0092] In the temporal feedback structure, the residual propagation path sets a fixed span connection mechanism to calculate the difference between the input features of the current layer and the output features of the two layers above. The residual signal is generated by pixel-level difference summation to form a multi-channel residual information stream. The splicing operation combines the residual signal with the output features in the channel dimension, introducing structural memory. The gated recurrent unit sets a forget gate and input gate update mechanism to retain cross-layer information through the gradient propagation path. The feature representation sequence establishes a time step index according to the encoding depth to form a sequence representation structure with contextual continuity, improving the dynamic modeling capability of thread wear evolution and crack propagation trend.

[0093] In the residual correction rule, the semantic consistency metric is calculated from the semantic vectors of the residual signals before and after encoding. The semantic vectors are based on the global average pooling of the feature map and the output of the fully connected network. The semantic offset of the residual signal is calculated by weighting the similarity difference and the activation distribution overlap rate, and is set as the fusion metric score. If the fusion metric score is less than the preset similarity threshold, it is determined to be inconsistent in expression. Under this condition, the gradient adjustment operation is performed to correct the parameters of the previous layer through the backpropagation path. The weight update operation is based on the error backpropagation weights and the regularization term to jointly optimize the loss function and generate the feature map of the residual signal.

[0094] This implementation method can enhance the robust modeling ability for complex texture perturbations and edge structure changes in images, and effectively improve the accuracy and stability of fastener micro-defect identification.

[0095] In this embodiment, the improved TransNeXt model undergoes the following structural and functional optimizations based on the TransNeXt model to enhance its semantic consistency modeling capability and temporal feature representation capability in complex structured images. The main improvements include:

[0096] The TransNeXt model mainly uses a single-scale feature extraction mechanism, which is difficult to fully express the edge texture and structural hierarchy information in the image. The improved TransNeXt model constructs multiple scale branch paths and extracts edge and texture features under different receptive fields through asymmetric convolution and dilated convolution. It also integrates shallow details and deep semantics through a skip connection mechanism to enhance the model's multi-scale recognition ability for fastener images.

[0097] The attention mechanism in the TransNeXt model suffers from semantic drift when dealing with repetitive textures and interfering features. The improved model introduces semantic relation vectors into the attention matrix and dynamically calculates the dependencies between pixels through cosine similarity, which effectively improves the semantic consistency modeling capability and solves the misidentification phenomenon caused by feature similarity in the original model.

[0098] The TransNeXt model lacks a cross-layer dependency modeling mechanism between feature sequences, making it unable to effectively learn the semantic change process. The improved model introduces gated recurrent units and residual signal concatenation paths to achieve dynamic feedback of features between front and back layers, preserve the semantic evolution trajectory, and enhance the model's ability to express changes in complex defect stages.

[0099] The gradient propagation path in the TransNeXt model is fixed and lacks a correction mechanism for semantic offset. The improved model calculates the residual signal bias based on the semantic consistency metric and achieves semantic error self-calibration by adjusting the residual path weights and the parameters of the previous convolution layer, thereby improving the model's stable recognition ability of low-confidence regions.

[0100] This implementation method, while maintaining the structural advantages of the original model, systematically enhances the multi-scale texture representation, semantic consistency modeling, temporal feature feedback, and semantic residual correction through the improved TransNeXt model. This significantly improves the recognition robustness and classification accuracy in scenarios with complex fastener structures, strong texture interference, and obvious defect evolution.

[0101] In this embodiment, S3 specifically refers to:

[0102] Image feature maps are input into a hierarchical coding structure. Different convolution kernel sizes and dilation rates are set for each scale branch path to extract initial edge features and local texture features of the fastener image. In each branch path, an asymmetric convolution structure is used to calculate the orientation-sensitive response and extract geometric edge features in the horizontal and vertical directions. Dilated convolutions are used to capture changes in thread spacing and surface roughness within different receptive fields, forming a multi-scale feature layer. Skip connections are constructed between the feature layers at each scale, and feature concatenation and channel normalization are performed on shallow and deep features to fuse local texture and global structural information, generating a hierarchical coding result. The hierarchical coding result has the ability to express multi-scale fusion features, providing high-resolution feature input for subsequent semantic mapping.

[0103] The hierarchical encoding results are input into a dynamic mapping structure, and a dependency mapping feature map is calculated by combining semantic relation vectors. The semantic relation vectors are obtained by calculating the cosine similarity between the feature vectors of each pixel in the hierarchical encoding results. The similarity calculation is performed by fitting the cosine of the included angle to obtain the semantic relevance matrix. The dependency mapping feature map is generated by weighting the semantic relation vectors into the attention matrix. The weights of the attention matrix are dynamically updated by minimizing the residuals to enhance spatial structure relevance and semantic consistency. After generating the dependency mapping feature map, channel weighting and orthogonal mapping correction are performed on the feature map to obtain a semantic consistency enhancement result. The semantic consistency enhancement operation can suppress redundant responses in texture repetition regions and improve the semantic discrimination ability of defective regions.

[0104] The dependency mapping feature map is input into the temporal feedback structure to establish cross-layer feature dependencies. The temporal feedback structure calculates the difference between the initial predicted feature and the output feature of the previous layer by setting the residual propagation path. The residual difference is fitted to obtain the residual signal by a pixel-level weighted summation function. The output feature of the previous layer and the residual signal are concatenated in the channel dimension and input into the gated recurrent unit for feature state update. The gated recurrent unit sets the weights of the forget gate and the update gate so that the feature state forms a memory sequence in the time dimension. The feature expression sequence maintains the semantic continuity and temporal consistency of the features before and after through cross-layer residual concatenation.

[0105] The residual correction rule is applied to the feature representation sequence, and the semantic consistency metric of the residual signals of the previous and next layers is calculated. The semantic consistency metric is obtained by cosine fitting the global average pooling vector of the residual signal feature map with the feature encoding vector of the current layer. If the consistency score is less than a preset threshold, gradient adjustment and weight update operations are performed to correct the convolution parameters and mapping weight matrix of the previous layer. The corrected feature representation sequence is then weighted and combined to form the residual signal feature map.

[0106] The residual signal feature map is input into the fully connected layer and the classification output layer. Based on multidimensional semantic feature encoding, defect stage labels, identification confidence and feature encoding vectors are generated. The feature encoding vector is obtained through high-dimensional semantic mapping of the residual signal feature map, which contains spatial structure information and semantic dependency features.

[0107] This implementation method can simultaneously preserve the geometric edge features and texture details of the fastener surface during the deep feature extraction process, enhancing the model's ability to express complex structural defects. Through temporal feedback and residual correction mechanisms, it improves the model's stability and generalization performance under multi-scale semantic mapping, achieving high-confidence identification and accurate classification of different defect stages of fasteners.

[0108] In this embodiment, S4 specifically refers to:

[0109] Based on the defect stage label, determine the location range of the fastener in the abnormal stage, and construct a confidence screening mask on the image feature map in combination with the identification confidence. Call the element-level screening function to mark the image area with the identification confidence greater than the preset suspicious threshold as a suspicious area.

[0110] For the suspicious region, a feature extraction operation is performed in the feature encoding vector to extract the encoding channel containing semantic residuals. The channel is then input into a multi-channel convolutional structure containing three parallel convolutional kernels to decode the texture structure, edge direction, and semantic boundary within the suspicious region, generating a region-level feature map.

[0111] The region-level feature map is upsampled, and the nearest neighbor interpolation method is used to restore the feature map to the same spatial scale as the original image. The boundary correction path is called to perform position alignment and edge fitting operations on the region boundary to correct the edge deviation of the region-level feature map and obtain the corrected decoding result in the image scale space.

[0112] The fusion correction decoding result is combined with the original image structure-guided edge map. The edge map is extracted from the original image based on the gradient-guided method. The fusion operation adopts a channel-weighted average mechanism, where the weight coefficients are obtained by fitting a structural similarity measurement function. The fusion result is used to construct a confidence weight map.

[0113] The confidence weight maps are superimposed along the channel dimension, and a two-dimensional grayscale response map is output using a linear superposition rule. The grayscale response map is then standardized and mapped to a heat map space to generate a defect response heat map. The defect response heat map includes regional response intensity distribution and location activation information.

[0114] In this embodiment, S5 specifically refers to:

[0115] A channel fusion operation is performed on the defect response heatmap. The multi-channel thermal response values ​​are combined proportionally using a weighted average fusion method. The fusion weight is calculated by the channel confidence fitting function to generate a single-channel response intensity map.

[0116] In the single-channel response intensity map, the local extremum scanning module is called to extract local peak points in the neighborhood of all pixels, and regional connectivity analysis is performed to identify continuous regions with complete response structures. A response intensity threshold is set to filter high-response regions, which are sets of pixels with pixel values ​​greater than the response intensity threshold and spatially connected structures. The response intensity threshold is obtained by combining the Otsu threshold method with confidence distribution fitting.

[0117] Boundary fitting is performed on each high-response region. The minimum bounding rectangle calculation method is used to extract the region's bounding boundary. The center coordinates, width and height dimensions, and rotation angle parameters of the rectangle are output. The above parameters are recorded to construct a candidate boundary set.

[0118] Feature extraction is performed on the response intensity values ​​within each candidate bounding box. The average response value, maximum response value, and response variance of the pixels contained therein are calculated to obtain the response distribution characteristics of the region corresponding to the bounding box. The response variance is fitted and corrected using an unbiased estimation method.

[0119] The response distribution features of each candidate bounding box are jointly modeled with its geometric morphological features. A high-dimensional description vector is constructed by calling a multi-dimensional vector combination function and input into the feature consistency judgment mechanism to filter out bounding boxes whose morphological features and response features do not match, thereby generating a set of defect candidate regions. The set of defect candidate regions includes the boundary parameters, morphological description information and response statistical features of each region.

[0120] In this embodiment, the step of performing channel fusion on the defect response heatmap to generate a single-channel response intensity map specifically involves:

[0121] Normalization is performed on each channel of the defect response heatmap to standardize the pixel values ​​within the channel to a set range;

[0122] A pixel-wise weighted summation operation is performed on the normalized multi-channel heatmap, wherein the weighting coefficients are set according to the channel importance of the decoding convolution kernel corresponding to each channel;

[0123] The weighted summation fused image is used as a single-channel response intensity image, and the weighted response value of each pixel in all channel responses is retained;

[0124] Edge smoothing is performed on the single-channel response intensity map, and Gaussian filtering is used to suppress local noise fluctuations.

[0125] Output a single-channel response intensity map, which includes the fused response value of each image region in the multi-channel response.

[0126] In this embodiment, S6 specifically refers to:

[0127] Geometric morphology analysis is performed on the bounding box of each candidate region in the defect candidate region set. The boundary contour extraction path is called to obtain the contour point set. The aspect ratio, contour complexity, and boundary smoothness are calculated based on the coordinates of the point set. The aspect ratio is calculated by the ratio of the width to the height of the bounding rectangle of the boundary. The contour complexity is obtained by fitting the curvature change rate of the contour points. The boundary smoothness is calculated by the root mean square of the second derivative.

[0128] Set up geometric shape filtering rules, define the aspect ratio range, complexity upper limit and smoothness lower limit parameters, mark candidate regions that do not meet the three constraints as abnormal shape regions and remove them, and obtain the set of regions after geometric filtering.

[0129] For the retained candidate regions, a connectivity judgment operation is performed. A connected graph structure is constructed based on the pixel neighborhood relationship. A depth-first search algorithm is used to traverse the connected regions and count the number and area ratio of connected components. The presence of isolated pixel blocks or broken connection regions in the connected graph is detected. Candidate regions with connectivity less than a preset connectivity threshold are removed, and a set of connected regions is generated.

[0130] For each region in the set of connected regions, a response anomaly index is calculated. The response anomaly index includes the region's average response deviation, response gradient distribution, and response extreme value fluctuation. The region's average response deviation is calculated by the absolute value of the difference between the region's response value and the global average response value. The response gradient distribution is obtained by fitting a gradient histogram. The response extreme value fluctuation is calculated by the variance of local extreme points. All indices are fused using a weighted average to obtain a comprehensive response anomaly value.

[0131] Set a response anomaly threshold, perform a rejection operation on candidate regions whose response anomaly is less than the threshold, and retain high-confidence regions that meet the threshold conditions to form a filtered candidate set.

[0132] Candidate regions that simultaneously meet the geometric shape filtering rules, connectivity judgment criteria, and response anomaly evaluation criteria are marked as the final defect identification results; the final defect identification results include region boundary information, response feature parameters, and connectivity measures.

[0133] In this embodiment, S7 specifically refers to:

[0134] For each defect region in the final defect identification result, the location coordinate information is extracted, and the boundary extraction path is called to obtain the coordinates of the region's center point and the coordinates of the bounding box vertices. The center point coordinates are calculated using the bounding box geometric center formula, and the bounding box vertex coordinates are obtained by outputting the rectangle vertex position matrix.

[0135] The area estimate is calculated based on the boundary range of the defect area and the pixel size ratio. The area value is obtained by linear fitting of the pixel number and the actual size ratio factor. The ratio factor is determined based on the resolution of the imaging device and the lens focal length calibration results, and is expressed in a unified physical unit.

[0136] Based on the label results of the corresponding defect area in the classification output layer, label the defect type name and establish a defect type set; index and number the same type of defect type to generate a type index table for statistical use.

[0137] The identification confidence level of the statistical defect region in the feature expression sequence is calculated. The maximum value in the confidence vector is extracted as the identification confidence value. The identification confidence level is calculated through the confidence distribution function to form a set of identification confidence levels.

[0138] By integrating defect type, location coordinates, area estimate, and identification confidence level, a statistical information table is constructed using structured splicing rules. The statistical information table uses a row-column mapping structure to record the comprehensive attribute information of each defect, including type field, location field, area field, and confidence level field.

[0139] Based on the defect location coordinates recorded in the statistical information table, defect marking boundaries are drawn in the original image. Different colors are used to identify different defect types, and defect type and identification confidence level information are superimposed on the boundaries.

[0140] Example 1:

[0141] To verify the feasibility of this invention in practice, it was applied to three typical maintenance production lines at a city rail transit operation and maintenance base as test subjects. The method of this invention was deployed to collect and process bolt image data in actual assembly scenarios. The application period was three months, using a self-built data acquisition device for on-site image acquisition, collecting approximately 1200 images per day, for a total of 108,000 images. The test included multiple dimensions such as defect detection accuracy, false detection rate, recognition time, and manual inspection efficiency.

[0142] During implementation, the acquired images are input into the preprocessing module for size standardization and reflection suppression. Subsequently, local peak values ​​in continuous regions are extracted using the response intensity map, and a response intensity threshold is set to filter high-response areas. Feature sequences are extracted from high-response areas and defect state vectors are matched. The candidate regions are input into the improved TransNeXt model, which utilizes its multi-scale hierarchical coding structure and time-dependent feedback path to accurately model and classify the boundary gaps, color difference corrosion, and offset contours of the bolt area. Then, the structured extraction process is called to obtain the defect location coordinates, area estimation, confidence level identification, and visualization annotation.

[0143] Table 1. Comparison of Defect Detection Performance of Bolt Fasteners under Different Algorithms

[0144] Indicator Name Rule-based edge detection method CNN model detection methods TransNeXt Model Method Method of the present invention Accuracy (%) 78.9 89.5 93.4 96.2 False positive rate (%) 12.6 6.3 4.8 2.3 Average recognition time (ms) 165.3 122.7 109.1 98.5 Minor defect identification rate (%) 49.2 63.8 71.3 84.7

[0145] As can be seen from the horizontal comparison results in Table 1, the method of this invention outperforms existing methods in all key indicators. Traditional rule-based edge detection methods are limited by fixed thresholds, resulting in an accuracy of less than 80% and a false detection rate as high as 12.6%, performing extremely poorly under varying lighting conditions and complex backgrounds. Although CNN models improve feature extraction capabilities through deep convolutional structures, they have low sensitivity to fine-grained defects, leading to a small defect recognition rate of only 63.8%. The original TransNeXt model has improved in structural feature modeling, but still suffers from insufficient utilization of temporal information. The improved TransNeXt model of this invention enhances multi-scale feature expression capabilities by introducing hierarchical coding and dynamic mapping structures, and improves feature stability by utilizing temporal feedback structures. This results in a detection accuracy of 96.2%, a false detection rate reduced to 2.3%, an average recognition time shortened to 98.5 ms, and a significant improvement in the small defect recognition rate to 84.7%, demonstrating a significant overall performance improvement and meeting the dual requirements of high precision and high real-time performance in industrial settings.

[0146] Table 2 Comparison of Detection Efficiency of Maintenance Personnel over Three Months

[0147] Detection methods Image processing volume Defect detection volume Misjudged number of interventions Manual review rate (%) Traditional manual-assisted detection 320 38 17 100 Based on the TransNeXt model 870 76 7 28.5 Method of the present invention 1180 91 3 11.4

[0148] Table 2 illustrates the impact of different methods on operational efficiency over a three-month period. Traditional manual inspection relies on manual observation, with an average daily processing volume of only 320 images, fewer than 40 defects detected, and as many as 17 misjudgments requiring intervention. The manual review rate remains at 100%, resulting in extremely low efficiency. Using the original TransNeXt model enhances automated recognition capabilities, increasing the processing volume to 870 images / day. However, due to the model's instability in recognizing some defect boundaries, 28.5% manual review is still required. The method of this invention optimizes the semantic consistency of feature representation through residual correction rules and temporal feedback structures, enabling the system to maintain stable recognition performance even under high complexity. The average daily image processing volume increases to 1180 images, with only 3 misjudgments requiring intervention, and the manual review rate drops to 11.4%. Overall operational efficiency improves by approximately 3.7 times, while reducing manpower input by nearly 90%, verifying the significant improvement in automation, efficiency, and accuracy of this method in real-world engineering scenarios.

[0149] This embodiment verifies that the method of the present invention outperforms existing methods in terms of stability, speed, and ability to detect small defects in bolt fasteners under actual industrial conditions, and has significant engineering practical value and promising prospects for technology promotion.

[0150] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatic identification of fastener defects based on deep learning, characterized in that, Includes the following steps: S1. Acquire fastener image data and perform size normalization, illumination equalization and noise suppression processing on the image data to generate standardized image samples; S2. Perform edge feature enhancement and multi-scale filtering operations on the standardized image samples to extract geometric edge structure, texture details and gray-level distribution features, and generate image feature maps; S3. Input the image feature map into the improved TransNeXt model and output the defect stage label, recognition confidence and feature encoding vector; the improved TransNeXt model includes a hierarchical encoding structure, a dynamic mapping structure, a temporal feedback structure and residual correction rules; S4. Based on the defect stage label and identification confidence level, filter the location of suspicious areas, and combine the feature encoding vector to perform local area decoding operation to generate a defect response heatmap; S5. Extract the bounding box, morphological features and response intensity distribution of the maximum response region based on the defect response heatmap, and construct a set of defect candidate regions; S6. Perform geometric morphology filtering, region connectivity judgment and response anomaly evaluation on the set of defect candidate regions to screen out non-defect regions and obtain the final defect identification result. S7. Based on the final defect identification results, construct a defect distribution map and statistical information table, and output the defect type, location coordinates, area estimate and identification confidence level.

2. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S1 is: Acquire fastener surface image data, including RGB images taken from different angles, distances, and lighting conditions; The image data is subjected to size normalization processing to unify the image size to a preset resolution range and maintain a consistent aspect ratio; the size-normalized image is subjected to illumination equalization processing, and the brightness distribution is corrected using the histogram equalization method. Noise suppression processing is performed on the image after illumination equalization. Median filtering and bilateral filtering algorithms are used in combination to optimize high-frequency noise and edge detail preservation, respectively, and to remove background interference and imaging errors in image acquisition. The processed image is saved as a standard format image file to construct standardized image samples.

3. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S2 is: Edge feature enhancement processing is performed on standardized image samples. Gradient operators are used to extract high-frequency edge information in the image, and an adaptive threshold segmentation method is combined to highlight the fastener outline and thread structure line. Multi-scale filtering is performed on the edge-enhanced image, and Gaussian and Laplacian filter layers with different kernel sizes are constructed to extract features from the detail layer and the structural layer in a hierarchical manner. In the multi-scale feature extraction process, the difference mapping between feature layers at each scale is calculated, and the local texture change region and the gray-scale distribution abnormal region are weighted and superimposed to generate a fused feature map; normalization and channel concatenation operations are performed on the fused feature map to form an image feature map.

4. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, The improved TransNeXt model is specifically as follows: The improved TransNeXt model includes a hierarchical coding structure, a dynamic mapping structure, a temporal feedback structure, and a residual correction rule; The image feature map is divided into channels using a hierarchical coding structure, and multiple scale branch paths are set. Each scale branch path uses asymmetric convolution kernels and dilated convolution structures to extract geometric edge features and local texture features. Skip connection paths are constructed to fuse the shallow and deep features of each scale branch to generate hierarchical coding results. The dynamic mapping structure constructs an attention distribution map based on the hierarchical encoding results, introduces semantic relation vectors, performs weight guidance operations on the attention matrix, and generates a dependency mapping feature map that integrates semantic consistency and spatial relevance. The temporal feedback structure establishes a cross-layer residual propagation path between each coding layer, calculates the difference between the initial predicted feature and the output feature of the previous layer, and generates a residual signal; the output feature of the previous layer and the residual signal are concatenated and input to the next layer, and the feature state is updated through a gated loop unit to form a feature expression sequence; the feature expression sequence has a time dependency. The residual correction rule constructs a semantic consistency metric based on the residual signals of the preceding and following coding layers to determine the degree of deviation of the residual signal in semantic expression; if it is less than the preset similarity threshold, gradient adjustment and weight update operations are performed on the residual signal to generate a residual signal feature map.

5. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S3 is: The image feature map is input into a hierarchical coding structure, and initial edge features and texture features are extracted on each scale branch path respectively; Cross-layer feature fusion is performed by skipping connection paths to generate hierarchical coding results; The hierarchical encoding result is input into the dynamic mapping structure, and the dependency mapping feature map is calculated by combining the semantic relation vector. Then, the semantic consistency enhancement operation is performed based on the attention distribution map. The dependency mapping feature map is input into a temporal feedback structure to construct a feature representation sequence; The feature representation sequence establishes cross-layer feature dependencies through residual signal splicing and gated update operations; The residual correction rule is applied to the feature expression sequence to generate a residual signal feature map, the degree of semantic expression shift is determined, and gradient adjustment and weight update are performed. The corrected feature representation sequence is input into the fully connected layer and the classification output layer to generate defect stage labels, recognition confidence and feature encoding vectors; The feature encoding vector is the semantic encoding result in the feature map of the residual signal.

6. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S4 is: Based on the defect stage label, determine the location range of the fastener in the abnormal stage, and construct a confidence screening mask on the image feature map in combination with the recognition confidence. Extract the image area with the recognition confidence greater than the preset suspicious threshold as the suspicious area. For the suspicious region, feature extraction is performed in the feature encoding vector, and a multi-channel convolutional structure is called to perform local semantic decoding to generate a region-level feature map; the region-level feature map is upsampled and the boundary is corrected to restore the feature map to the image scale space; A confidence weight map is constructed by fusing the region decoding results in the image scale space with the original image structure-guided edge map. The confidence weight maps are overlaid by channel to generate a defect response heatmap; The defect response heatmap includes regional response intensity distribution and location activation information.

7. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S5 is: Perform channel fusion on the defect response heatmap to generate a single-channel response intensity map; Local peak points of continuous regions are extracted from the single-channel response intensity map, and a response intensity threshold is set to filter high response regions; the high response region is a region with a pixel value greater than the response intensity threshold and forming a connected structure. Calculate the bounding box of the circumscribed rectangle based on the high response region, record the center coordinates, width and height dimensions and rotation angle of the bounding box, and construct a candidate boundary set; Extract the response intensity distribution of the region corresponding to each bounding box in the candidate boundary set, calculate the average response value, maximum response value and response variance, and obtain the response distribution characteristics; By combining response distribution characteristics and bounding box geometry, a set of candidate defect regions is constructed; the set of candidate defect regions includes region boundary information, morphological description information and response statistical characteristics.

8. The method for automatic identification of fastener defects based on deep learning according to claim 7, characterized in that, The process of performing channel fusion on the defect response heatmap to generate a single-channel response intensity map is as follows: Normalization is performed on each channel of the defect response heatmap to standardize the pixel values ​​within the channel to a set range; A pixel-wise weighted summation operation is performed on the normalized multi-channel heatmap, wherein the weighting coefficients are set according to the channel importance of the decoding convolution kernel corresponding to each channel; The weighted summation fused image is used as a single-channel response intensity image, and the weighted response value of each pixel in all channel responses is retained; Edge smoothing is performed on the single-channel response intensity map, and Gaussian filtering is used to suppress local noise fluctuations. Output a single-channel response intensity map, which includes the fused response value of each image region in the multi-channel response.

9. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S6 is: Perform geometric morphology analysis on the bounding box of each candidate region in the defect candidate region set, and calculate the aspect ratio, contour complexity and boundary smoothness; Set geometric shape filtering rules to eliminate candidate regions that do not meet the set shape parameter range; Perform connectivity judgment on the retained candidate regions, construct a connected graph structure, and detect whether there are isolated pixel blocks or broken connection regions in the region; Candidate regions with connectivity less than a preset connectivity threshold are removed, and a set of connected regions is constructed. For each region in the set of connected regions, a response anomaly index is calculated, which includes the region's average response deviation, response gradient distribution, and response extreme value fluctuation; candidate regions with a response anomaly index less than a preset response anomaly threshold are removed. Candidate regions that simultaneously meet the geometric shape filtering rules, connectivity judgment criteria, and response anomaly evaluation criteria are taken as the final defect identification results.

10. The method for automatic identification of fastener defects based on deep learning according to claim 1, characterized in that, Specifically, S7 is: Extract the location coordinate information of each defect region in the final defect identification result, including the coordinates of the region center point and the coordinates of the bounding box vertices; Based on the boundary range of the defect area and the pixel size ratio, the area estimate is calculated and expressed in a uniform unit. Based on the label results of the corresponding defect areas in the classification output layer, label the defect type names and construct a defect type set; The identification confidence level of the defective region in the feature expression sequence is statistically analyzed, and the identification confidence level value is recorded to form an identification confidence level set. By integrating defect type, location coordinates, area estimate, and identification confidence level, a structured statistical information table is constructed; Based on the defect location coordinates recorded in the statistical information table, draw defect label boundaries in the original image, overlay defect type and identification confidence level information, and construct a defect distribution map.

Images