Surface detection method and system for precise fastener

Abstract

The invention relates to the technical field of surface detection, and discloses a surface detection method and system for a precision fastener, and the method comprises the steps: carrying out the multi-angle image collection and image preprocessing of the precision fastener, and obtaining an initial feature image set; three-domain feature extraction is carried out, and surface feature fingerprint vectors are obtained through calculation of a texture feature domain, a shape feature domain and a material feature domain; constructing a depth separable convolution structure and a feature pyramid structure to obtain a depth surface detection network; taking the pre-trained target detection model as a teacher network, taking the deep surface detection network as a student network, and obtaining a target surface detection model through calculation of a characteristic distillation loss function and a response distillation loss function; and inputting a to-be-detected precise fastener image into the target surface detection model, and obtaining category information and position information of surface defects through two-stage measurement recognition calculation. According to the method, the calculation efficiency is improved while the detection precision of the precise fastener is ensured.

Description

Technical Field

[0001] The present application relates to the field of surface detection technology, and in particular to a surface detection method and system for precision fasteners. Background Art

[0002] Precision fasteners are key components in aerospace, and their surface quality directly affects the safety and reliability of equipment. With the development of intelligent manufacturing technology, the surface inspection of precision fasteners has gradually shifted from manual inspection to automated inspection to meet the needs of high-precision and high-efficiency inspection.

[0003] At present, traditional precision fastener surface inspection mainly relies on image processing and machine vision technology, and collects inspection data by setting up multiple sensors. However, this method has problems such as low inspection accuracy, poor adaptability, and low inspection efficiency. Especially under complex working conditions, due to factors such as changes in lighting conditions and surface reflections, it is easy to cause inaccurate inspection results. Summary of the invention

[0004] The present application provides a surface detection method and system for precision fasteners, which improves computational efficiency while ensuring detection accuracy of the precision fasteners.

[0005] In a first aspect, the present application provides a surface detection method for a precision fastener, the surface detection method for a precision fastener comprising: Multi-angle image acquisition and image preprocessing are performed on precision fasteners, and feature enhancement is performed through maximum reflection prior and weighted variational regularization model to obtain the initial feature image set; Performing three-domain feature extraction on the initial feature image set, and obtaining a surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; Based on the surface feature fingerprint vector, a deep separable convolution structure and a feature pyramid structure are constructed, and a deep surface detection network is obtained through channel shuffling operation and feature fusion operation; Using the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtaining a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; The image of the precision fastener to be inspected is input into the target surface inspection model, and the category information and position information of the surface defects are obtained through two-stage measurement recognition calculation.

[0006] A second aspect of the present application provides a surface detection system for a precision fastener, the surface detection system for a precision fastener comprising: The acquisition module is used to acquire and preprocess multi-angle images of precision fasteners, and enhance features through maximum reflection prior and weighted variational regularization models to obtain an initial feature image set; An extraction module is used to perform three-domain feature extraction on the initial feature image set, and obtain a surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; A construction module is used to construct a deep separable convolution structure and a feature pyramid structure based on the surface feature fingerprint vector, and obtain a deep surface detection network through channel shuffling operation and feature fusion operation; A calculation module, used to use the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtain a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; The recognition module is used to input the image of the precision fastener to be detected into the target surface detection model, and obtain the category information and position information of the surface defects through two-stage measurement recognition calculation.

[0007] A third aspect of the present application provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned surface detection method for precision fasteners.

[0008] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned surface detection method for precision fasteners.

[0009] Compared with the prior art, the present application has the following beneficial effects: through multi-angle image acquisition and image preprocessing technology, combined with maximum reflectance prior and weighted variational regularization model, the interference caused by illumination changes and surface reflections is effectively overcome, and the expression ability of image features is enhanced; a three-domain feature extraction method is adopted, and texture, shape and material features are comprehensively considered to construct a comprehensive surface feature fingerprint vector, which improves the comprehensiveness and robustness of feature expression; a lightweight network based on deep separable convolution and feature pyramid structure is designed, and through channel shuffling operations and feature fusion operations, the model calculation amount is significantly reduced while ensuring detection accuracy; a knowledge distillation mechanism is introduced, and through feature distillation loss function and response distillation loss function, the effective transfer of teacher network knowledge to student network is realized, and the model performance is further optimized; a two-stage metric recognition strategy is adopted to improve the computational efficiency while ensuring detection accuracy; the present invention can be quickly deployed on edge computing devices to meet the real-time detection needs of precision fasteners. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] The structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0012] Figure 1 It is a schematic flow chart of a surface detection method for a precision fastener provided by an embodiment of the present invention; Figure 2 is a schematic block diagram of the structure of a surface detection system for precision fasteners provided by an embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0015] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0016] It should be further understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. Figure 1 , an embodiment of the surface detection method of the precision fastener in the embodiment of the present application includes: Step S100, performing multi-angle image acquisition and image preprocessing on the precision fasteners, and performing feature enhancement through maximum reflection prior and weighted variational regularization model to obtain an initial feature image set; It is understandable that the execution subject of the present application may be a surface detection system for precision fasteners, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0017] Specifically, the images of precision fasteners are collected at four angles of 0°, 45°, 90°, and 135° to obtain complete surface information, covering various possible forms of defect manifestations. The collected original image data is color corrected to eliminate the influence of light and environment on the image and improve the quality and accuracy of the image. In the color correction process, the Shades of Grey color correction method is used to achieve color balance and correction by calculating the p-norm of the three RGB channels. The p-norm calculation is performed on the RGB channels of the original image data respectively, and the color correction factor is determined through this process to achieve balanced adjustment of the overall color of the image and obtain the color-corrected image data. The color correction method of Shades of Grey is used to minimize the color difference in the image under different lighting conditions, so that the subsequent feature extraction is more consistent and accurate. Based on the color-corrected image data, a maximum reflection prior model is constructed. This model aims to further explore and enhance the reflective characteristics of the fastener surface. By calculating the weighted average of the local reflectivity and global reflectivity of the image, the reflectivity feature mapping data is obtained, which can effectively highlight the potential defect areas. The reflectance prior model can enhance the focus on target features in complex backgrounds by combining local information with global information, making the reflectance features more prominent and improving the recognition of defects. The reflectance feature map data is subjected to weighted variational regularization calculation to smooth the image while retaining important edge information. Appropriate regularization parameters are set and the calculation is performed through iterative optimization to obtain the enhanced feature image. The enhanced feature image is subjected to brightness normalization and linear transformation. Brightness normalization is used to eliminate the brightness differences caused by different angles and lighting conditions, so that the overall brightness distribution of the image tends to be consistent. The linear transformation adjusts the dynamic range of the image to make its feature information more prominent. After brightness normalization and linear transformation, a standardized feature image is obtained. The standardized feature image is subjected to contrast enhancement processing to highlight the detailed features in the image, making the differences between different regions more obvious and improving the recognition of defects. The feature images that have been subjected to contrast enhancement are subjected to feature fusion at different angles. The feature information at different angles is integrated to generate a more comprehensive and accurate feature description. By fusing information from multiple angles, the detection blind spots caused by a single viewing angle are effectively reduced, the overall detection performance is improved, and a multi-angle fusion feature image is obtained. The feature image after multi-angle fusion is subjected to noise suppression processing to remove the residual random noise in the image and obtain the initial feature image set.

[0018] Step S200, performing three-domain feature extraction on the initial feature image set, and obtaining a surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; Specifically, the initial feature image set is divided into multiple small pixel blocks. The gray value distribution of each pixel block is analyzed and encoded using the local binary pattern (LBP). The features at different scales are extracted through the LBP feature pyramid, and the texture feature matrix is ​​generated by combining the statistical calculation of the gray level co-occurrence matrix (GLCM). Based on the generated texture feature matrix, wavelet multi-resolution decomposition is used to process the features. The image is decomposed into high-frequency and low-frequency sub-bands through wavelet decomposition to capture the detailed features of the image at different scales. The coefficients of the sub-bands are statistically analyzed by the histogram of oriented gradients (HOG) to describe the edge and gradient distribution characteristics in the image. At the same time, the Sobel edge detection operator is combined for processing to effectively extract the edge information in the image and obtain the shape feature matrix. The shape feature matrix is ​​input into the Gabor filter group for convolution kernel calculation. The Gabor filter group filters the responses of different frequencies and directions, and can effectively extract the texture and direction information of the surface. The material feature matrix is ​​obtained by extracting the local energy and local phase of the filtered response features. The texture feature matrix, shape feature matrix and material feature matrix are cross-domain adaptively weighted fused to form a three-domain feature tensor. Through cross-domain fusion, the three features complement each other, so that the model can comprehensively utilize different feature information when detecting surface defects of precision fasteners to achieve better detection results. The three-domain feature tensor is constructed with triplet samples to improve the distinguishing ability of the model. The distance threshold α of the positive sample pair and the distance threshold β of the negative sample pair are set, and the triplet feature pairs are constructed by selecting anchor samples, positive samples and negative samples. The triplet loss function value is calculated based on the triplet feature pairs. By maximizing the Euclidean distance difference between the positive and negative sample pairs and performing feature normalization, the feature similarity loss is obtained, so that the model can better distinguish features of different categories, ensure that samples of the same category are closer in the feature space, and samples of different categories are farther away, thereby improving the accuracy of detection. At the same time, in order to optimize the feature expression, the three-domain feature tensor is adaptively clustered, and the feature center of each category is dynamically updated by calculating the intra-class feature variance and inter-class feature distance to obtain the center loss function value. The intra-class feature variance is used to describe the compactness of samples of the same type in the feature space, while the inter-class feature distance is used to evaluate the degree of separation between different classes. By minimizing the intra-class variance and maximizing the inter-class distance, we can ensure that the various features are reasonably distributed in the feature space, which helps to improve the generalization performance of the detection model. The feature similarity loss and the center loss function value are jointly optimized to improve the robustness and discrimination ability of the model and obtain the surface feature fingerprint vector.

[0019] The L2 norm normalization is performed on the three-domain feature tensor, and the feature vector is standardized to unit length, thereby eliminating the difference in magnitude of different features and ensuring that all samples have the same weight in the feature space. After completing the L2 norm normalization, the Euclidean distance matrix between sample pairs is calculated based on the feature vector of unit length to construct the distance matrix. Threshold segmentation is performed based on the distance matrix to divide the positive and negative sample pairs. By setting the distance threshold α of the positive sample pair and the distance threshold β of the negative sample pair, each pair of samples in the distance matrix is ​​marked as a positive sample pair or a negative sample pair. A positive sample pair refers to a sample pair whose Euclidean distance between the feature vectors is less than the positive sample pair distance threshold α, while a negative sample pair refers to a sample pair whose Euclidean distance is greater than the negative sample pair distance threshold β. Anchor samples are selected based on the labeling matrix. Anchor samples are benchmark samples for comparison and play a core role in triplet samples. In order to ensure that the anchor samples can represent the entire sample space, the distance distribution between each sample and other samples is calculated, and the samples with the most uniform distance distribution with other samples are selected to ensure that the anchor samples can cover the key areas in the entire feature space and obtain the anchor sample set. Positive sample matching is performed on the anchor sample set to find sample pairs that are close to the anchor sample. These samples meet the condition that the Euclidean distance is less than the positive sample pair distance threshold α. By finding all sample pairs that meet this condition and screening them according to the principle of minimum distance, a positive sample set is obtained. At the same time, a negative sample set is determined. By selecting sample pairs that are far away from the anchor sample, that is, sample pairs whose distance is greater than the negative sample pair distance threshold β, and performing hard sample mining on this basis, a negative sample set is obtained. The hard sample mining process can help the model better learn challenging negative samples, thereby improving the model's discrimination ability. The anchor sample set, positive sample set, and negative sample set are combined to construct triplet feature pairs. Through splicing operations and index alignment, the three samples are combined so that they can be input into the loss function together for optimization. Based on the triplet feature pairs, the triplet loss function is calculated to measure the accuracy of the model by comparing the distance between the positive sample pair and the anchor sample, and the distance between the negative sample pair and the anchor sample. The core of the triplet loss function calculation is to calculate the weighted difference between the distance of the positive sample pair and the distance of the negative sample pair by setting the marginal parameter. The introduction of the margin parameter enables the system to set a reasonable "interval" between the positive and negative sample pairs, thereby ensuring that the distance between the positive sample pair and the anchor sample is as small as possible, while the distance between the negative sample pair and the anchor sample is as large as possible. The purpose of weighted difference calculation is to enable the model to maximize the distance difference between the positive and negative sample pairs in the feature space, thereby improving the model's discriminative ability and enabling it to better identify surface defects. The feature similarity loss is obtained.

[0020] Step S300, constructing a deep separable convolution structure and a feature pyramid structure based on the surface feature fingerprint vector, and obtaining a deep surface detection network through channel shuffling operation and feature fusion operation; It should be noted that the surface feature fingerprint vector is input into a 5-layer depthwise separable convolution structure. Each layer of the depthwise separable convolution structure contains a 3×3 depthwise convolution layer and a 1×1 pointwise convolution layer. The depthwise convolution layer is used to extract spatial features, and the pointwise convolution layer is used to compress and expand the features. After each layer, the ReLU activation function and batch normalization layer are added to increase the nonlinear expression ability of the model and effectively alleviate the gradient vanishing problem during training. Through this structural design, a multi-layer convolution feature map is obtained, which contains the deep information of the input surface feature fingerprint vector. The multi-layer convolution feature map is input into the feature pyramid network for processing. The feature pyramid network contains five levels with different resolutions, namely P1, P2, P3, P4 and P5, where the resolution of P1 is 32×32, P2 is 16×16, P3 is 8×8, P4 is 4×4, and P5 is 2×2. Through the multi-level structure, the feature pyramid network can capture the feature information of the image at different scales and better adapt to surface defects of various sizes. The multi-scale feature representation is fused from top to bottom. The feature fusion is performed by 2x upsampling and lateral skip connection, combining high-level semantic information with low-level spatial information to obtain a fused feature atlas. The fused feature atlas is input into the channel shuffle module. The channel shuffle module divides the feature channels into 4 groups, reorders the channels in each group and exchanges features between groups to obtain shuffled feature maps. Through the channel shuffle operation, the model can effectively increase the information exchange between channels, making the information contained in each channel richer and more diverse, and improving the ability of feature expression. The shuffled feature map is subjected to residual connection operation. The residual connection operation can effectively solve the common gradient vanishing problem in deep networks and promote the flow of information in the network by adding short-circuit connections between each depth-separable convolutional layer. In order to ensure the effect of residual connection, the residual scale factor is set to 0.1, so that in the process of residual connection, the original feature information can be balanced with the feature information after convolution operation to obtain optimized feature representation. According to the optimized feature representation, a dual-branch detection head is constructed. The dual-branch detection head contains three 3×3 convolutional layers for category prediction and four 3×3 convolutional layers for position regression. Each convolutional layer is followed by BatchNorm and ReLU activation functions to improve the learning ability of the convolutional layer and promote feature normalization. Through these convolutional layers, the network can simultaneously output the category information and position regression information of surface defects to obtain the detection output features. Based on the detection output features, the 5-layer deep separable convolutional structure, feature pyramid network, channel shuffle module and dual-branch detection head are combined into a deep surface detection network.

[0021] Step S400: using the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtaining a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; Specifically, the output of the backbone feature extraction network in the pre-trained object detection model is used as the teacher feature, and the features are reorganized to obtain the multi-scale feature map of the teacher network. The pre-trained object detection model includes a backbone feature extraction network and a multi-scale feature enhancement module, wherein the multi-scale feature enhancement module contains three parallel branches: spatial attention branch, channel attention branch and position sensitive branch. Each branch is designed to enhance different types of feature representations, and improve the network's ability to capture features through attention mechanism and position sensitivity, thereby helping the teacher network to better extract fine surface features. Channel reconstruction is performed on the feature maps of layers P2 to P4 of the deep surface detection network. In order to make the multi-scale feature maps of the student network consistent with the teacher network, these feature maps are adjusted using 1×1 convolution operations, and their channel dimensions are adjusted to the same as the feature maps of the teacher network, ensuring that the student network can be consistent with the teacher network at the feature level. After channel reconstruction, the multi-scale feature map of the student network is obtained. Based on the multi-scale feature maps of the teacher network and the student network, a feature distillation loss function is constructed. The feature-level distillation loss is obtained by calculating the L2 distance between the teacher network and the student network at the same feature position. Feature distillation enables the student network to learn the rich information of the teacher network in the feature extraction process, thereby improving the feature learning ability of the student network. The calculation of L2 distance can quantify the difference between the two, and by minimizing the loss, the student network is gradually approached to the feature representation of the teacher network. Response-level distillation is calculated. The detection output layer of the pre-trained object detection model is used as the teacher response, and the category prediction probability is softened by the softmax function to obtain the softened teacher response. The softening process is to reduce the constraints of hard labels on the student network training, so that the student network can learn the knowledge of the teacher network more flexibly. The detection output of the deep surface detection network is softened, and the cross entropy between the softened output of the student network and the softened teacher response is calculated to obtain the response-level distillation loss. The feature-level distillation loss and the response-level distillation loss are combined to construct a joint optimization target. By setting a weighted combination of feature weights and response weights, the importance of feature distillation and response distillation in the optimization process is balanced to obtain the total distillation loss. The total distillation loss is back-propagated with gradients. By calculating the gradient of the loss function relative to the network parameters, the parameters of the student network are optimized so that it can better learn the knowledge of the teacher network. In the process of gradient back propagation, the parameters are updated layer by layer and the optimized parameters are loaded into the deep surface detection network. Through this step, the student network gradually acquires the knowledge of the teacher network and applies it to the defect detection task of the surface of precision fasteners. Through layer-by-layer parameter updating and model storage, the optimized target surface detection model is obtained.

[0022] Step S500: input the image of the precision fastener to be inspected into the target surface inspection model, and obtain the category information and location information of the surface defects through two-stage measurement recognition calculation.

[0023] Specifically, the image of the precision fastener to be inspected is input into the target surface inspection model. Through the forward calculation of the five-layer deep separable convolution structure and the P1 to P5 feature pyramid network, a multi-scale feature map is obtained, which contains surface information at different scales and can capture different feature levels from coarse to fine. Channel reorganization and feature aggregation are performed on the multi-scale feature map. Channel reorganization and feature aggregation are performed on the features of each scale through top-down upsampling and lateral connection. The feature map of each scale is adjusted to 256 channels by 1×1 convolution to ensure the unification of feature dimensions, so that subsequent feature fusion and calculation can be performed in the same dimensional space. A global feature vector is constructed based on the feature representation of unified dimensions. The feature map of each scale is pooled by adaptive average pooling operation to extract global feature information, and then the fully connected layer is used to calculate to obtain a normalized detection feature vector. The cosine similarity between the normalized detection feature vector and the defect category feature library is calculated. The similarity value between the detection feature vector and each category feature vector is obtained by calculating the vector inner product and dividing it by the modulus length, so as to assign a preliminary category prediction to the image to be inspected and obtain a preliminary candidate category set. Attention weighting is performed on the preliminary candidate category set. Through the weighted operation of spatial attention and channel attention, the product of the spatial attention map and the channel attention vector is calculated to improve the model's attention to important feature areas and enhance the recognition ability of defect categories to obtain the optimized candidate categories. Based on the optimized candidate categories, fine measurement calculation is performed, and the defect category information is obtained through the combined measurement method of local sensitive hashing and Euclidean distance. Local sensitive hashing can effectively process high-dimensional features and calculate the similarity for each candidate category to screen out the most likely defect category, while Euclidean distance is used to measure the difference between candidate categories and help make the final category judgment. Through the combination of the two, the model can more accurately identify the defect category and exclude irrelevant candidate categories. According to the defect category information, the corresponding position prediction branch is extracted in the dual-branch detection head network. The detection head network includes a category prediction branch and a position regression branch. Position regression accurately predicts the bounding box coordinates of the defect area through the cascade of four 3×3 convolutional layers and coordinate regression calculation. The bounding box coordinates of the defect area are subjected to maximum suppression processing to remove bounding boxes with high overlap, ensure that only one bounding box is retained for each defect area, and eliminate redundant predictions to obtain the category information and location information of the surface defects.

[0024] In the embodiments of the present application, through multi-angle image acquisition and image preprocessing technology, combined with maximum reflectance prior and weighted variational regularization model, the interference caused by illumination changes and surface reflections is effectively overcome, and the expression ability of image features is enhanced; a three-domain feature extraction method is adopted, and texture, shape and material features are comprehensively considered to construct a comprehensive surface feature fingerprint vector, which improves the comprehensiveness and robustness of feature expression; a lightweight network based on deep separable convolution and feature pyramid structure is designed, and through channel shuffling operations and feature fusion operations, the model calculation amount is significantly reduced while ensuring detection accuracy; a knowledge distillation mechanism is introduced, and through feature distillation loss function and response distillation loss function, the effective transfer of teacher network knowledge to student network is realized, and the model performance is further optimized; a two-stage metric recognition strategy is adopted to improve computational efficiency while ensuring detection accuracy; the present invention can be quickly deployed on edge computing devices to meet the real-time detection needs of precision fasteners.

[0025] In a specific embodiment, the process of executing step S100 may specifically include the following steps: The image of precision fasteners is collected at four angles of 0°, 45°, 90°, and 135° to obtain the original image data, and the original image data is subjected to Shades of Grey color correction calculation. The color-corrected image data is obtained by calculating the p-norm of the RGB three channels and determining the color correction factor; A maximum reflectance a priori model is constructed based on the color-corrected image data, and reflectance feature mapping data is obtained by calculating the weighted average of the local reflectance and the global reflectance of the image; Perform weighted variational regularization calculation on the reflectivity feature map data, and obtain the enhanced feature image by setting regularization parameters and iterative optimization calculation; Performing brightness normalization processing and linear transformation on the enhanced feature image to obtain a standardized feature image, and performing contrast enhancement processing on the standardized feature image to obtain a feature image after contrast enhancement; The feature images after contrast enhancement are subjected to feature fusion according to different angles to obtain multi-angle fusion feature images, and noise suppression is performed on the multi-angle fusion feature images to obtain an initial feature image set.

[0026] Specifically, the fasteners are imaged at different angles using Four angles are used to capture various features of the fastener surface and obtain original image data containing information from different perspectives. Shades of Grey color correction calculation is performed on the original image data to correct the color deviation caused by different lighting and angle changes. The image data is processed in three RGB channels. Assuming that the original image is ,in Represents the red, green and blue channels of the image respectively. In order to perform color correction, the p-norm is used to calculate and quantify the difference between the three RGB channels. The calculation formula is as follows: ; in, Represents the RGB value of each pixel in the image, is the order of the norm, is the total number of pixels in the image. This step obtains the p-norm of the image, thereby determining a color correction factor to further correct the image so that the color of each channel is more consistent and the color deviation under different angles and lighting is reduced. After color correction, the image data obtained is expressed as , which is more consistent with the real surface color than the original image. The maximum reflection a priori model is constructed based on the corrected image data to help the system capture the light reflection characteristics of the surface of precision fasteners, thereby enhancing the detection ability of surface details. The local reflectivity and global reflectivity of the image are calculated. These two reflectivities represent the light reflection intensity of the image in the local area and the global range, respectively. Assume that the brightness of each pixel in the image is ,in and represents the coordinates of the pixel, and the local reflectivity is expressed as: ; in, It is the maximum brightness in the image. The local reflectivity represents the ratio of the brightness of a pixel to the global brightness. The global reflectivity is calculated by the average value of the global brightness of the image. The formula is: ; in, is the number of pixels in the image, Pixel The local reflectivity and global reflectivity are combined by weighted average to obtain the reflectivity feature map data: ; in, and are the weight coefficients of local and global reflectivity, which are obtained through empirical values ​​or adaptive learning. Reflectivity feature map data It can effectively enhance the light reflection characteristics of the image. The reflectivity feature map data is calculated by weighted variational regularization. Variational regularization is an optimization method that improves the clarity of image features and reduces noise interference by adjusting regularization parameters. Weighted variational regularization is used to enhance the image, and the optimization target is expressed by the following formula: ; in, represents the gradient of the image, is the regularization parameter, is a smoothed image, which represents the denoised image. Through iterative optimization calculation, the enhanced feature image is obtained, noise and unnecessary details are removed, and the important features in the image are highlighted. Based on the enhanced feature image, brightness normalization processing and linear transformation are performed to obtain a standardized feature image. The influence of brightness changes of different images on the detection results is eliminated. The brightness normalization parameter of the image is set to and , normalized by the following formula: ; in, is the mean brightness of the image, is the standard deviation of brightness, is the enhanced feature image. The standardized image is subjected to contrast enhancement processing to improve the contrast of the image and make the details more obvious. The contrast of the image is enhanced by the linear transformation method, and the formula is: ; in, and is the adjustment parameter of contrast enhancement. Through contrast enhancement, the details of the image are more prominent. Feature fusion is performed on the feature images after contrast enhancement according to different angles to enhance the expression of surface features. By splicing and weighting images from different angles, a multi-angle fused feature image is obtained. Noise suppression is performed on the multi-angle fused feature image, and smoothing filtering or adaptive denoising technology is used to remove noise in the image to obtain the final initial feature image set.

[0027] In a specific embodiment, the process of executing step S200 may specifically include the following steps: The initial feature image set is segmented into pixel blocks to obtain multiple pixel blocks, and the texture feature matrix is ​​obtained by LBP feature pyramid extraction and GLCM texture matrix statistics based on the gray value distribution and local binary pattern calculation of each pixel block; Based on the texture feature matrix, wavelet multi-resolution decomposition is performed, and the shape feature matrix is ​​obtained by calculating the directional gradient histogram statistics of the high-frequency and low-frequency sub-band coefficients and the Sobel edge detection operator. The shape feature matrix is ​​input into the Gabor filter bank for convolution kernel calculation to obtain the response features after filtering, and the local energy and local phase of the response features after filtering are extracted to obtain the material feature matrix; Perform cross-domain adaptive weighted fusion on texture feature matrix, shape feature matrix and material feature matrix to obtain a three-domain feature tensor; The triplet samples of the three-domain feature tensor are constructed. By setting the distance threshold α of the positive sample pair and the distance threshold β of the negative sample pair, anchor samples, positive samples and negative samples are selected to obtain triplet feature pairs. The triplet loss function value is calculated based on the triplet feature pairs. The feature similarity loss is obtained by maximizing the Euclidean distance difference between the positive and negative sample pairs and performing feature normalization. Adaptive feature clustering is performed on the three-domain feature tensor. The feature center of each category is updated by calculating the intra-class feature variance and the inter-class feature distance to obtain the center loss function value. The feature similarity loss and the center loss function value are jointly optimized to obtain the surface feature fingerprint vector.

[0028] Specifically, the initial feature image set is segmented into pixel blocks to obtain multiple small pixel blocks, thereby locally processing the image, enhancing the surface detail features, and helping to reduce the interference of background noise. Image segmentation is performed by setting a fixed-size sliding window or an adaptive segmentation method. The size of each pixel block is selected according to the density of information and the demand for computing resources. After the segmentation is completed, the gray value distribution of each pixel block is calculated, and texture features are extracted based on the local binary pattern (LBP). LBP is a commonly used texture description operator that generates a binary value by comparing the gray value of each pixel with its neighboring pixels. Given a 3×3 neighborhood, the gray value of the central pixel is compared with the gray value of the surrounding 8 neighboring pixels. If the gray value of the neighboring pixel is greater than that of the central pixel, it is recorded as 1, otherwise it is recorded as 0. This process generates an 8-bit binary number, which is converted into a decimal number to obtain the feature value of the local binary pattern. For each pixel block, its LBP image is calculated and extracted through the LBP feature pyramid. The LBP feature pyramid is a hierarchical feature extraction method that captures the texture details of the image through LBP encoding at different scales. The mathematical representation of this process is: ; in, is the number of pixels in the neighborhood, Represents the gray value of a pixel. is a sign function, indicating whether the current pixel is larger than the pixels in the surrounding neighborhood. After GLCM (gray level co-occurrence matrix) statistics, richer texture features are extracted. Through GLCM calculation, the texture of the image is quantified through the gray level co-occurrence relationship in different directions and distances to obtain a texture matrix. The calculation formula is: ; in, Indicates the distance and direction Next, Pixel and pixels The grayscale co-occurrence probability. Through the statistical analysis of these co-occurrence matrices, a series of texture features are obtained, such as contrast, energy, homogeneity, etc. Based on the texture feature matrix, wavelet multi-resolution decomposition is performed. Wavelet transform is a method that can simultaneously analyze between the time domain and the frequency domain, and can effectively extract high-frequency and low-frequency components in the signal. In image processing, discrete wavelet transform is used to decompose the image at multiple scales to obtain sub-band coefficients at different resolutions. Assume that the image is the original image, and the coefficients after wavelet transformation are expressed as: ; ; in, and are low-pass filter and high-pass filter respectively. and Represent the low-frequency and high-frequency subband coefficients respectively. The directional gradient histogram statistics of the high-frequency and low-frequency subband coefficients are performed, and the Sobel operator is combined for edge detection to obtain the shape feature matrix. The Sobel operator can effectively detect the edge information in the image by convolving the image and calculating the gradient of the image. The calculation formula is: ; in, and are the gradients of the image in the x and y directions respectively. This method is used to extract the shape features in the image. The shape feature matrix is ​​input into the Gabor filter bank for convolution operation. The Gabor filter is a filter that can provide local feature analysis in both the spatial and frequency domains, and can effectively extract the texture and structural information of the image. Assume that the shape feature matrix of the image is , the output of the Gabor filter is expressed as: ; in, is the standard deviation of the filter, is the frequency, is the direction angle. After convolution operation through Gabor filter, the filter response characteristics are obtained Contains local texture information of the image. By extracting the local energy and local phase information of the filtered response features, the material feature matrix is ​​obtained. These material features describe the reflection characteristics, lighting effects, etc. of the surface material in the image. After extracting the features of texture, shape and material in three dimensions, these feature matrices are cross-domain adaptive weighted fusion. Through weighted fusion, the feature information of the three fields is comprehensively utilized to obtain a multi-dimensional and comprehensive surface feature representation. The calculation of weighted fusion is implemented by the following formula: ; in, Represent the texture, shape and material feature matrices respectively, is the weight of the corresponding field. By adjusting the weight coefficient, the features of different fields play different roles. The triplet sample is constructed for the three-domain feature tensor. The triplet sample includes anchor samples, positive samples and negative samples. The classification effect is enhanced by maximizing the Euclidean distance difference between positive and negative samples. Assume that the Euclidean distance between positive samples and negative samples is , by optimizing the following loss function to minimize the distance between negative pairs and maximize the distance between positive pairs: ; in, is a marginal threshold. The feature representation is optimized by calculating the feature similarity loss and the center loss. The feature similarity loss maximizes the difference in Euclidean distance between sample pairs, while the center loss optimizes the feature center of each category by calculating the intra-class feature variance and the inter-class feature distance. The surface feature fingerprint vector is obtained by jointly optimizing the feature similarity loss and the center loss function.

[0029] In a specific embodiment, the execution step constructs a triplet sample for the three-domain feature tensor, sets a positive sample pair distance threshold α and a negative sample pair distance threshold β, selects anchor samples, positive samples, and negative samples, obtains triplet feature pairs, and calculates the triplet loss function value based on the triplet feature pairs. The process of obtaining the feature similarity loss by maximizing the Euclidean distance difference between the positive and negative sample pairs and performing feature normalization processing can specifically include the following steps: The three-domain feature tensors are normalized by L2 norm to obtain feature vectors of unit length, and the Euclidean distance matrix between sample pairs is calculated based on the feature vectors of unit length to obtain a distance matrix; Perform threshold segmentation on the distance matrix, obtain the labeling matrix of positive and negative sample pairs according to the distance threshold α of the positive sample pair and the distance threshold β of the negative sample pair, and select anchor samples based on the labeling matrix. By calculating the distance distribution between each sample and other samples, select the sample with the most uniform distance distribution to obtain the anchor sample set; The anchor sample set is matched with positive samples. By finding sample pairs whose distance is less than the positive sample pair distance threshold α and screening them according to the minimum distance principle, a positive sample set is obtained. At the same time, negative samples are determined based on the positive sample set. By selecting sample pairs whose distance is greater than the negative sample pair distance threshold β and performing hard sample mining, a negative sample set is obtained. The anchor sample set, positive sample set and negative sample set are combined to construct triplet feature pairs. Batched triplet features are obtained through splicing operation and index alignment. The triplet loss is calculated based on the batched triplet features. The weighted difference between the positive sample pair distance and the negative sample pair distance is calculated by setting the marginal parameter to obtain the feature similarity loss.

[0030] Specifically, the three-domain feature tensor is normalized by L2 norm to obtain a feature vector of unit length. The length of each feature vector is normalized to 1, so that the feature vector only contains directional information and eliminates its scale difference. Given a feature vector f, its L2 norm normalization process is expressed as: ; in, Representation vector The L2 norm of Represents the normalized eigenvector. Based on the eigenvector of unit length, the Euclidean distance matrix between sample pairs is calculated. Assume that samples, and the feature vector of each sample is , then the Euclidean distance between sample pairs is expressed as: ; in, Representation sample and samples The Euclidean distance between and Respectively represent samples and samples In the Through this step, we get The distance matrix ,in Representation sample and samples The Euclidean distance between them. Perform threshold segmentation on the distance matrix. By setting the distance threshold of the positive sample And the negative sample distance threshold , divide the distance matrix into positive sample pairs and negative sample pairs. , then the sample and samples is marked as a positive sample pair; if , then the sample and samples are marked as negative sample pairs. Based on these labels, the label matrix is ​​constructed ,in represents a positive sample pair, represents a negative sample pair, Indicates that the distance between the two does not meet the threshold requirement for positive and negative sample pairs. Anchor samples are selected based on the label matrix. Anchor samples refer to samples that are representative in the feature space. Samples with the most uniform distance distribution are usually selected as anchor samples. When selecting anchor samples, the distance distribution between each sample and other samples is calculated. Assume that the sample The distance distribution is , using variance to measure the uniformity of its distance distribution. Sample The variance of is expressed as: ; in, It is a sample The mean distance of Representation sample By calculating the variance of all samples and selecting the sample with the smallest variance as the anchor sample, we get the anchor sample set . Perform positive sample matching on the anchor sample set. For each anchor sample , find the distance less than the positive sample distance threshold samples and screen them based on the principle of minimum distance. ,and In all and Distance less than The smallest sample, then the sample As positive samples and anchor samples Pairing. Thus we get a positive sample set . Determine negative samples based on the positive sample set. For each positive sample , select a distance greater than the negative sample distance threshold The samples of are regarded as negative samples. , then the sample Selected as negative samples. Through hard sample mining, we ensure that the most difficult negative samples are selected, thereby improving the model's discrimination ability. Get the negative sample set . Combine the anchor sample set, positive sample set and negative sample set to construct triplet feature pairs. Each triplet sample contains an anchor sample, a positive sample and a negative sample. Through concatenation and index alignment, these samples are combined into batched triplet features, expressed as: ; For these triplet features, training is performed based on the triplet loss function. The goal of the triplet loss function is to maximize the similarity between the positive sample pair and the anchor sample, while minimizing the similarity between the negative sample pair and the anchor sample. The triplet loss function is expressed as: ; in, and Respectively represent the Euclidean distance between the anchor sample and the positive sample, and between the anchor sample and the negative sample, is the marginal parameter. In order to better train the model, a weighted difference is calculated between the distances of positive and negative samples to obtain the final feature similarity loss: ; in, and is a weighting coefficient used to adjust the relative impact of positive and negative samples on the loss. By jointly optimizing these loss functions, an optimized surface feature fingerprint vector is obtained, which can effectively represent and distinguish different defect categories when performing surface defect detection.

[0031] In a specific embodiment, the process of executing step S300 may specifically include the following steps: The surface feature fingerprint vector is input into a 5-layer depthwise separable convolution structure. Each layer of the depthwise separable convolution structure contains a 3×3 depthwise convolution layer and a 1×1 pointwise convolution layer. A ReLU activation function and a batch normalization layer are added after each layer to obtain a multi-layer convolution feature map. Input the multi-layer convolutional feature map into the feature pyramid network, which includes five levels: P1, P2, P3, P4, and P5. The resolution of P1 is 32×32, the resolution of P2 is 16×16, the resolution of P3 is 8×8, the resolution of P4 is 4×4, and the resolution of P5 is 2×2, to obtain multi-scale feature representation; Perform top-down feature fusion on multi-scale feature representations, combine high-level semantic information with low-level spatial information through 2x upsampling and lateral skip connections, and obtain a fused feature atlas; The fused feature atlas is input into the channel shuffling module, and the feature channels are divided into 4 groups through the channel shuffling module. Channels are rearranged in each group and features are exchanged between groups to obtain the shuffled feature map. Perform residual connection operation on the shuffled feature map, add short-circuit connection between each depth-separable convolutional layer, and set the residual scaling factor to 0.1 to obtain the optimized feature representation; A dual-branch detection head is constructed based on the optimized feature representation. The dual-branch detection head contains three 3×3 convolutional layers for category prediction and four 3×3 convolutional layers for position regression. Each convolutional layer is followed by BatchNorm and ReLU activation functions to obtain the detection output features. Based on the detection output features, a 5-layer deep separable convolutional structure, a feature pyramid network, a channel shuffle module and a dual-branch detection head are combined into a deep surface detection network.

[0032] Specifically, the surface feature fingerprint vector is input into a network consisting of five layers of depthwise separable convolutional structures. Each layer of depthwise separable convolutional structure consists of two main parts: a 3×3 depthwise convolutional layer and a Point-wise convolutional layer. The function of the deep convolutional layer is to process the spatial information of the input features, while the point-wise convolutional layer is used to mix features in the channel dimension. Each convolutional layer is followed by a ReLU activation function and a batch normalization layer, which helps to speed up the training of the network and improve the generalization ability of the model. Given the input feature , No. The output of the layer is represented as: ; Among them, DepthConv represents depth convolution, PointConv represents point-by-point convolution, is the output of the previous layer, is the output of the current layer. Through this structure, the network is able to extract rich features at multiple levels. The multi-layer convolutional feature map is input into the feature pyramid network, which has five levels of different resolutions: P1, P2, P3, P4 and P5, corresponding to different resolutions: 32×32, 16×16, 8×8, 4×4 and At this stage, the feature pyramid network captures multi-scale features from high-level semantic information to low-level spatial information through feature representations at multiple resolutions. The output feature map of the layer is , the output feature maps from P1 to P5 are: ; The feature pyramid network upsamples and laterally connects feature maps of different scales to generate feature representations of different resolutions, including all features from low-level spatial information to high-level semantic information. Top-down feature fusion is performed on multi-scale feature representations. The spatial information of low-level features and the semantic information of high-level features are combined to obtain a richer feature expression. Each high-level feature is upsampled (for example, using a 2×2 upsampling factor), and then the upsampled high-level features are fused with the low-level features through lateral jump connections. The fused feature atlas It is expressed as: ; in, Represents the feature concatenation operation, and UpSample is the upsampling operation. In this way, the network combines rich spatial information and semantic information at different scales. The fused feature atlas is input into the channel shuffling module, and the channel distribution of the feature map is made more uniform by rearranging and exchanging feature channels, thereby improving the expressiveness of the model. In this module, the channels of the feature map are divided into four groups, and the channels are rearranged within each group, and features are exchanged between groups. The channel shuffling operation helps to retain important information in the spatial dimension while enhancing the complementarity between channels. Residual connection operations are performed on the shuffled feature maps. Between each depth-separable convolutional layer, a short-circuit connection (i.e., residual connection) is added so that the input of each layer can be directly passed to the next layer, thereby alleviating the gradient vanishing problem and accelerating the training process. The residual connection is implemented in the following way: ; in, is the residual scaling factor, which is usually small (e.g., 0.1). In this way, the features of each layer can retain the input features of the previous layer while avoiding information loss. Based on the optimized feature representation, a dual-branch detection head is constructed to perform defect category prediction and location regression at the same time. The category prediction branch consists of three The position regression branch consists of 4 convolutional layers, each of which is followed by batch normalization and ReLU activation function. The deep surface detection network is constructed by combining a five-layer deep separable convolutional structure, a feature pyramid network, a channel shuffle module, and a dual-branch detection head.

[0033] In a specific embodiment, the process of executing step S400 may specifically include the following steps: The output of the backbone feature extraction network in the pre-trained target detection model is used as the teacher feature, and the features are reorganized to obtain the multi-scale feature map of the teacher network; the pre-trained target detection model includes a backbone feature extraction network and a multi-scale feature enhancement module, and the multi-scale feature enhancement module includes three parallel branches: spatial attention branch, channel attention branch and position sensitive branch; The feature maps of layers P2 to P4 of the deep surface detection network are reconstructed through channels and adjusted to the same channel dimension as the teacher network through 1×1 convolution to obtain the multi-scale feature maps of the student network; A feature distillation loss function is constructed based on the multi-scale feature maps of the teacher network and the multi-scale feature maps of the student network, and the feature-level distillation loss is obtained by calculating the corresponding L2 distance; The detection output layer of the pre-trained object detection model is used as the teacher response, and the category prediction probability is softened by the softmax function to obtain the softened teacher response; The detection output of the deep surface detection network is softened, and the response-level distillation loss is obtained by calculating the cross entropy between the softened teacher response; A joint optimization objective is constructed based on the feature-level distillation loss and the response-level distillation loss. The total distillation loss is obtained by setting a weighted combination of feature weights and response weights. The total distillation loss is back-propagated with gradients to obtain the optimized network parameters, which are then loaded into the deep surface detection network. The target surface detection model is obtained through layer-by-layer parameter updating and model storage.

[0034] Specifically, the output of the backbone feature extraction network in the pre-trained target detection model is used as the feature input of the teacher network. The design of the teacher network includes two main components: the backbone feature extraction network and the multi-scale feature enhancement module. The backbone feature extraction network is used to extract the underlying spatial features from the input image, while the multi-scale feature enhancement module enhances the expressiveness of these features through three parallel branches: spatial attention, channel attention, and position sensitivity. The spatial attention branch highlights the key areas by learning the weights of the spatial position; the channel attention branch assigns weights according to the importance of the channel; and the position sensitive branch flexibly adapts in spatial position to improve the model's robustness to spatial changes. The output feature map of the teacher network is represented as: SpatialAttention ChannelAttention PositionSensitive ; in, It is the initial feature map obtained from the input image through the backbone feature extraction network. Represents the concatenation of the features of the three branches. For the student network, the feature maps of layers P2 to P4 in the deep surface detection network need to be channel reconstructed so that they are aligned with the feature maps of the teacher network in the channel dimension. Convolution adjusts the feature map to have the same channel dimension as the teacher network. In this way, the student network is able to learn similar feature representations as the teacher network. The output feature map of the layer is , then through The feature map after convolution is expressed as: ; Among them, Conv represents the convolution operation, and the adjusted number of channels matches the feature map of the teacher network. In order to improve the learning ability of the student network, feature distillation technology is used. The training of the student network is guided by calculating the difference in feature maps between the teacher network and the student network. The L2 distance between the multi-scale feature maps of the teacher and student networks is calculated to measure their differences. The feature-level distillation loss is expressed as: ; in, and The teacher and student networks are The feature map of the layer, Denotes the L2 norm. The feature-level distillation loss encourages the feature map of the student network to be as close as possible to the feature map of the teacher network, thereby transmitting useful learning signals. The detection output layer of the pre-trained object detection model is used as the teacher response, and the category prediction probability is softened by the softmax function to obtain the softened teacher response. Assume that the category prediction result of the teacher network is , then the softened teacher response is:

[0035] in, is the category prediction logits of the teacher network, and Softmax represents the normalized softmax function, which is used to convert logits into probability distribution. Similarly, the detection output of the student network is softened. Assume that the category prediction result of the student network is , then the student's softening response is: ; The response-level distillation loss is implemented by calculating the cross entropy between the student and the teacher: ; in, and The teacher and student networks are Prediction probability on category. In order to comprehensively consider the feature distillation loss and response distillation loss, a joint optimization objective is constructed. A weighted combination of feature-level distillation loss and response-level distillation loss is set to form the total distillation loss: ; in, and is the weight coefficient of feature distillation loss and response distillation loss, which controls the relative importance of the two in the total loss. Gradient backpropagation is performed based on the total distillation loss to optimize the parameters of the student network. By updating the parameters of each layer and loading the optimized parameters into the deep surface detection network, a target surface detection model optimized by distillation learning is obtained. By updating the parameters layer by layer, the model can gradually approach the performance of the teacher network, thereby improving the detection accuracy of the student network.

[0036] In a specific embodiment, the process of executing step S500 may specifically include the following steps: The image of the precision fastener to be inspected is input into the target surface detection model, and a multi-scale feature map is obtained through the forward calculation of the five-layer depth-separable convolution structure and the P1 to P5 feature pyramid network; Channel reorganization and feature aggregation are performed on multi-scale feature maps. Through top-down upsampling and lateral connections, the features of each scale are adjusted to 256 channels through 1×1 convolution to obtain feature representation of unified dimension. A global feature vector is constructed based on the feature representation of unified dimensions, and a normalized detection feature vector is obtained through adaptive average pooling and fully connected layer operations; The cosine similarity between the normalized detection feature vector and the defect category feature library is calculated, and the preliminary candidate category set is obtained by dividing the vector inner product by the modulus length; Perform attention weighting on the preliminary candidate category set, obtain the optimized candidate category by calculating the product of the spatial attention map and the channel attention vector, and perform fine metric calculation based on the optimized candidate category, and obtain the defect category information through the combined metric method of local sensitive hashing and Euclidean distance; According to the defect category information, the corresponding position prediction branch is extracted in the dual-branch detection head network. The bounding box coordinates of the defect area are obtained through the cascade of four 3×3 convolutional layers and coordinate regression calculation. The bounding box coordinates of the defect area are then subjected to maximum suppression processing to obtain the category information and position information of the surface defects.

[0037] Specifically, the image of the precision fastener to be inspected is input into the target surface detection model for processing. Forward calculation is performed through a five-layer depth-separable convolution structure and a P1 to P5 feature pyramid network to obtain a multi-scale feature map. Each layer of the depth-separable convolution structure contains a 3×3 depth convolution layer and a 1×1 point-by-point convolution layer. The depth convolution layer extracts local features in space, while the point-by-point convolution is used to reduce dimensions and increase nonlinearity. Each layer of convolution is followed by a ReLU activation function and a batch normalization layer, which helps to improve the nonlinear expression ability of the network and the stability of training. Through the step-by-step processing of these levels, the features of the image will be gradually extracted and form multi-scale feature maps, which can effectively capture surface information of different scales. For the obtained multi-scale feature maps, the model needs to perform channel reorganization and feature aggregation. This process is achieved through top-down upsampling and lateral connections. In the output of each layer, the features of each scale are adjusted to the same number of channels (256 channels) through a 1×1 convolution operation. Channel reorganization helps to unify the representation of features at different scales, allowing the model to process multi-scale information in subsequent stages without losing any details. Through lateral connections, the high-level semantic information of each layer can be combined with the spatial information of the lower layer, enhancing the network's learning ability at multiple scales. Based on the feature representation of unified dimensions, a global feature vector is constructed. The model applies an adaptive average pooling method to pool each feature map and resize it to a fixed size so that feature maps of all scales can be input into a fully connected layer. The fully connected layer converts the pooled feature map into a one-dimensional global feature vector that carries the overall information of the input image. Through normalization operations, the feature vector is ensured to be more stable during training and the convergence of the model is improved. The normalized detection feature vector is expressed as: ; in, Indicates feature map, represents the L2 norm of the vector, is the final normalized global feature vector. The normalized detection feature vector is compared with the predefined defect category feature library. The feature library contains representative feature vectors for each category. The model determines the similarity between the input image and a defect category by calculating the cosine similarity between the vectors. The cosine similarity is calculated as: ; in, Yes Category The characteristic vector of is the normalized feature vector of the input image. The higher the cosine similarity, the greater the similarity between the input image and the category, and vice versa. Based on the calculated similarity, the model can select a preliminary set of candidate defect categories. Perform attention weighting on the preliminary candidate categories. Attention maps and attention vectors are generated by spatial attention and channel attention respectively, and the category selection is optimized by multiplying the feature map and the attention map. For example, the spatial attention map can highlight the key areas in the image, while the channel attention vector adjusts the importance of different channels. The optimized candidate categories are further optimized by fine metric calculation. Fine metric calculation measures the differences between candidate categories by combining local sensitive hashing and Euclidean distance. The model calculates the Euclidean distance of each pair of candidate categories and filters out the most relevant category information through local sensitive hashing. The Euclidean distance calculation formula is: ; After completing the defect category selection, the model extracts the corresponding position prediction branch in the dual-branch detection head based on the defect category information. The dual-branch detection head contains two branches: one for category prediction and the other for position regression. Each branch consists of multiple 3×3 convolutional layers to extract higher-level features and perform calculations. The category prediction branch outputs the probability of each category through convolution operations, while the position regression branch outputs the coordinates of the target bounding box. The calculation formula for category prediction is: ; in, Yes Category The prediction score is converted into a probability distribution by the Softmax function. For position regression, the network calculates the regression parameters of the target position (i.e., the coordinates of the bounding box) through four 3×3 convolutional layers. Through the regression model, the bounding box coordinates of the target are obtained. ,in are the center coordinates of the box, is the width and height of the box. In order to improve the accuracy of position prediction, the model uses the maximum suppression method to filter out the most appropriate bounding box. For multiple overlapping boxes in the detection results, the box with the highest confidence is retained and other overlapping boxes are removed. When the overlapping area (intersection over union, IoU) of two boxes is greater than the set threshold, the box with lower confidence will be suppressed. Through the above steps, the coordinates of the defect area bounding box are finally obtained, which is the location of the surface defect of the precision fastener, and the category information of the defect is obtained through category prediction.

[0038] The surface detection method of the precision fastener in the embodiment of the present application is described above. The surface detection system 10 of the precision fastener in the embodiment of the present application is described below. Figure 2 In one embodiment of the present application, a surface detection system 10 for precision fasteners includes: The acquisition module 11 is used to perform multi-angle image acquisition and image preprocessing on the precision fasteners, and perform feature enhancement through maximum reflection prior and weighted variational regularization model to obtain an initial feature image set; The extraction module 12 is used to perform three-domain feature extraction on the initial feature image set, and obtain the surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; A construction module 13 is used to construct a deep separable convolution structure and a feature pyramid structure based on the surface feature fingerprint vector, and obtain a deep surface detection network through channel shuffling operation and feature fusion operation; A calculation module 14 is used to use the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtain a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; The recognition module 15 is used to input the image of the precision fastener to be detected into the target surface detection model, and obtain the category information and position information of the surface defects through two-stage measurement recognition calculation.

[0039] Through the collaborative cooperation of the above-mentioned components, through multi-angle image acquisition and image preprocessing technology, combined with maximum reflectance prior and weighted variational regularization model, the interference caused by illumination changes and surface reflections is effectively overcome, and the expression ability of image features is enhanced; a three-domain feature extraction method is adopted, and texture, shape and material features are comprehensively considered to construct a comprehensive surface feature fingerprint vector, which improves the comprehensiveness and robustness of feature expression; a lightweight network based on deep separable convolution and feature pyramid structure is designed, and through channel shuffling operations and feature fusion operations, the model calculation amount is significantly reduced while ensuring detection accuracy; the knowledge distillation mechanism is introduced, and through the feature distillation loss function and the response distillation loss function, the effective transfer of teacher network knowledge to the student network is realized, and the model performance is further optimized; a two-stage metric recognition strategy is adopted to improve the computational efficiency while ensuring detection accuracy; the present invention can be quickly deployed on edge computing devices to meet the real-time detection needs of precision fasteners.

[0040] See also Figure 3 , Figure 3 This is a schematic block diagram of the structure of an electronic device 300 provided in an embodiment of the present application. The electronic device 300 includes a processor 301 and a memory 302. The processor 301 and the memory 302 are connected via a system bus 303, wherein the memory 302 may include a non-volatile storage medium and an internal memory.

[0041] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 301, the processor 301 can execute any of the above-mentioned surface detection methods for precision fasteners.

[0042] The processor 301 is used to provide computing and control capabilities to support the operation of the entire electronic device 300 .

[0043] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 301, the processor 301 can execute any of the above-mentioned surface detection methods for precision fasteners.

[0044] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the electronic device 300 involved in the present application scheme. The specific electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0045] It should be understood that the processor 301 may be a central processing unit (CPU), and the processor 301 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0046] It should be noted that those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the electronic device 300 described above can refer to the corresponding process of the surface detection method of the aforementioned precision fasteners, and will not be repeated here.

[0047] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the surface detection method of precision fasteners provided in the embodiment of the present application.

[0048] The computer-readable storage medium may be an internal storage unit of the electronic device 300 in the aforementioned embodiment, such as a hard disk or memory of the electronic device 300. The computer-readable storage medium may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the electronic device 300.

[0049] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0050] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0051] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A surface detection method for precision fasteners, characterized in that: The method comprises: Multi-angle image acquisition and image preprocessing are performed on precision fasteners, and feature enhancement is performed through maximum reflection prior and weighted variational regularization model to obtain the initial feature image set; Performing three-domain feature extraction on the initial feature image set, and obtaining a surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; Based on the surface feature fingerprint vector, a deep separable convolution structure and a feature pyramid structure are constructed, and a deep surface detection network is obtained through channel shuffling operation and feature fusion operation; Using the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtaining a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; The image of the precision fastener to be inspected is input into the target surface inspection model, and the category information and position information of the surface defects are obtained through two-stage measurement recognition calculation.

2. The surface detection method of precision fasteners according to claim 1, characterized in that: The method performs multi-angle image acquisition and image preprocessing on the precision fasteners, and performs feature enhancement through maximum reflection prior and weighted variational regularization model to obtain an initial feature image set, including: The image of the precision fastener is collected at four angles of 0°, 45°, 90°, and 135° to obtain the original image data, and the Shades of Grey color correction calculation is performed on the original image data. The color-corrected image data is obtained by calculating the p-norm of the RGB three channels and determining the color correction factor; Constructing a maximum reflectance a priori model based on the color-corrected image data, and obtaining reflectance feature mapping data by calculating a weighted average of local reflectance and global reflectance of the image; Performing weighted variational regularization calculation on the reflectivity feature map data, and obtaining an enhanced feature image by setting regularization parameters and iterative optimization calculation; Performing brightness normalization processing and linear transformation based on the enhanced feature image to obtain a standardized feature image, and performing contrast enhancement processing on the standardized feature image to obtain a contrast-enhanced feature image; The feature images after contrast enhancement are subjected to feature fusion according to different angles to obtain multi-angle fusion feature images, and noise suppression processing is performed on the multi-angle fusion feature images to obtain an initial feature image set.

3. The surface detection method of a precision fastener according to claim 1, characterized in that: The three-domain feature extraction is performed on the initial feature image set, and a surface feature fingerprint vector is obtained by calculating the texture feature domain, the shape feature domain and the material feature domain, including: The initial feature image set is segmented into pixel blocks to obtain a plurality of pixel blocks, and a texture feature matrix is ​​obtained by performing LBP feature pyramid extraction and GLCM texture matrix statistics according to the gray value distribution and local binary pattern calculation of each pixel block; Performing wavelet multi-resolution decomposition based on the texture feature matrix, and obtaining a shape feature matrix by calculating the directional gradient histogram statistics of the high-frequency and low-frequency sub-band coefficients and the Sobel edge detection operator; Inputting the shape feature matrix into a Gabor filter bank to perform convolution kernel calculation to obtain a filtered response feature, and performing local energy and local phase extraction on the filtered response feature to obtain a material feature matrix; Performing cross-domain adaptive weighted fusion on the texture feature matrix, the shape feature matrix and the material feature matrix to obtain a three-domain feature tensor; Performing triplet sample construction on the three-domain feature tensor, selecting anchor samples, positive samples and negative samples by setting a distance threshold α for positive sample pairs and a distance threshold β for negative sample pairs, obtaining triplet feature pairs, and calculating a triplet loss function value based on the triplet feature pairs, obtaining feature similarity loss by maximizing the Euclidean distance difference between the positive and negative sample pairs and performing feature normalization processing; Adaptive feature clustering is performed on the three-domain feature tensors. The feature center of each category is updated by calculating the intra-class feature variance and the inter-class feature distance to obtain the center loss function value. The feature similarity loss and the center loss function value are jointly optimized to obtain the surface feature fingerprint vector.

4. The surface detection method of a precision fastener according to claim 3, characterized in that: The triplet sample construction is performed on the three-domain feature tensor, by setting the positive sample pair distance threshold α and the negative sample pair distance threshold β, selecting anchor samples, positive samples and negative samples, obtaining triplet feature pairs, and calculating the triplet loss function value based on the triplet feature pairs, and obtaining the feature similarity loss by maximizing the Euclidean distance difference between the positive and negative sample pairs and performing feature normalization processing, including: Normalizing the three-domain feature tensor with an L2 norm to obtain a feature vector of unit length, and calculating a Euclidean distance matrix between sample pairs based on the feature vector of unit length to obtain a distance matrix; Perform threshold segmentation on the distance matrix, obtain a labeling matrix of positive and negative sample pairs according to the distance threshold α of the positive sample pair and the distance threshold β of the negative sample pair, select anchor samples based on the labeling matrix, calculate the distance distribution between each sample and other samples, select samples with the most uniform distance distribution, and obtain an anchor sample set; Performing positive sample matching on the anchor sample set, by finding sample pairs whose distance is less than the positive sample pair distance threshold α, and screening them according to the minimum distance principle, to obtain a positive sample set; at the same time, determining negative samples based on the positive sample set, by selecting sample pairs whose distance is greater than the negative sample pair distance threshold β, and performing hard sample mining, to obtain a negative sample set; The anchor sample set, the positive sample set and the negative sample set are combined to construct triplet feature pairs, batched triplet features are obtained through splicing operations and index alignment, and the triplet loss is calculated based on the batched triplet features. The weighted difference calculation is performed on the positive sample pair distance and the negative sample pair distance by setting marginal parameters to obtain the feature similarity loss.

5. The surface detection method of a precision fastener according to claim 1, characterized in that: The method of constructing a deep separable convolution structure and a feature pyramid structure based on the surface feature fingerprint vector, and obtaining a deep surface detection network through channel shuffling operation and feature fusion operation, includes: Inputting the surface feature fingerprint vector into a 5-layer depthwise separable convolution structure, each layer of the depthwise separable convolution structure includes a 3×3 depthwise convolution layer and a 1×1 pointwise convolution layer, and adding a ReLU activation function and a batch normalization layer after each layer to obtain a multi-layer convolution feature map; Input the multi-layer convolutional feature map into a feature pyramid network, including five levels P1, P2, P3, P4, and P5, where the resolution of P1 is 32×32, the resolution of P2 is 16×16, the resolution of P3 is 8×8, the resolution of P4 is 4×4, and the resolution of P5 is 2×2, to obtain a multi-scale feature representation; Performing top-down feature fusion on the multi-scale feature representation, combining high-level semantic information with low-level spatial information through 2-fold upsampling and lateral skip connection, and obtaining a fused feature atlas; Input the fused feature atlas into a channel shuffling module, divide the feature channels into 4 groups through the channel shuffling module, rearrange channels in each group and exchange features between groups to obtain a shuffled feature map; Performing a residual connection operation on the shuffled feature map, by adding a short-circuit connection between each depth-wise separable convolutional layer and setting the residual scaling factor to 0.1, to obtain an optimized feature representation; Constructing a dual-branch detection head according to the optimized feature representation, wherein the dual-branch detection head comprises three 3×3 convolutional layers for category prediction and four 3×3 convolutional layers for position regression, each convolutional layer is followed by a BatchNorm and a ReLU activation function to obtain a detection output feature; Based on the detection output features, the 5-layer depth-separable convolutional structure, the feature pyramid network, the channel shuffling module and the dual-branch detection head are combined into a deep surface detection network.

6. The surface detection method of a precision fastener according to claim 1, characterized in that: The method uses the pre-trained target detection model as a teacher network, uses the deep surface detection network as a student network, and obtains a target surface detection model by calculating a feature distillation loss function and a response distillation loss function, including: The output of the backbone feature extraction network in the pre-trained target detection model is used as the teacher feature, and the feature is reorganized to obtain a multi-scale feature map of the teacher network; the pre-trained target detection model includes a backbone feature extraction network and a multi-scale feature enhancement module, and the multi-scale feature enhancement module includes three parallel branches: a spatial attention branch, a channel attention branch, and a position-sensitive branch; Reconstruct the channels of the feature maps of layers P2 to P4 of the deep surface detection network, adjust them to the same channel dimension as the teacher network through 1×1 convolution, and obtain the multi-scale feature maps of the student network; Constructing a feature distillation loss function based on the multi-scale feature map of the teacher network and the multi-scale feature map of the student network, and obtaining a feature-level distillation loss by calculating the corresponding L2 distance; Using the detection output layer of the pre-trained target detection model as the teacher response, softening the category prediction probability through a softmax function to obtain a softened teacher response; Softening the detection output of the deep surface detection network, and obtaining the response-level distillation loss by calculating the cross entropy between the response and the softened teacher response; Constructing a joint optimization target according to the feature-level distillation loss and the response-level distillation loss, and obtaining a total distillation loss by setting a weighted combination of feature weights and response weights; The total distillation loss is subjected to gradient back propagation to obtain optimized network parameters, and the optimized network parameters are loaded into a deep surface detection network, and a target surface detection model is obtained through layer-by-layer parameter updating and model storage.

7. The surface detection method of a precision fastener according to claim 1, characterized in that: The image of the precision fastener to be inspected is input into the target surface inspection model, and the category information and location information of the surface defects are obtained through two-stage measurement recognition calculation, including: The image of the precision fastener to be inspected is input into the target surface detection model, and a multi-scale feature map is obtained through forward calculation of a five-layer depth-separable convolution structure and a P1 to P5 feature pyramid network; Channel reorganization and feature aggregation are performed on the multi-scale feature map. Through top-down upsampling and lateral connection, the features of each scale are adjusted to 256 channels through 1×1 convolution to obtain a feature representation of unified dimension; Constructing a global feature vector based on the feature representation of the unified dimension, and obtaining a normalized detection feature vector through adaptive average pooling and fully connected layer operations; Calculate the cosine similarity between the normalized detection feature vector and the defect category feature library, and obtain a preliminary candidate category set by dividing the vector inner product by the modulus length; Attention weighting is performed on the preliminary candidate category set, and an optimized candidate category is obtained by calculating the product of the spatial attention map and the channel attention vector, and a fine metric calculation is performed based on the optimized candidate category to obtain defect category information through a combined metric method of local sensitive hashing and Euclidean distance; According to the defect category information, the corresponding position prediction branch is extracted in the dual-branch detection head network, and the bounding box coordinates of the defect area are obtained through the cascade of four 3×3 convolutional layers and coordinate regression calculation. The bounding box coordinates of the defect area are subjected to maximum suppression processing to obtain the category information and position information of the surface defects.

8. A surface inspection system for precision fasteners, characterized in that: Used to perform the surface detection method of a precision fastener as described in any one of claims 1 to 7, the surface detection system of the precision fastener comprises: The acquisition module is used to acquire and preprocess multi-angle images of precision fasteners, and enhance features through maximum reflection prior and weighted variational regularization models to obtain an initial feature image set; An extraction module is used to perform three-domain feature extraction on the initial feature image set, and obtain a surface feature fingerprint vector by calculating the texture feature domain, the shape feature domain and the material feature domain; A construction module is used to construct a deep separable convolution structure and a feature pyramid structure based on the surface feature fingerprint vector, and obtain a deep surface detection network through channel shuffling operation and feature fusion operation; A calculation module, used to use the pre-trained target detection model as a teacher network and the deep surface detection network as a student network, and obtain a target surface detection model by calculating a feature distillation loss function and a response distillation loss function; The recognition module is used to input the image of the precision fastener to be detected into the target surface detection model, and obtain the category information and position information of the surface defects through two-stage measurement recognition calculation.

9. An electronic device, characterized in that: The electronic device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to execute the surface detection method for precision fasteners according to any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the surface detection method for precision fasteners according to any one of claims 1 to 7 is implemented.

Images